Method and system for determining a quality measure for an image using multi-level decomposition of images

ABSTRACT

Method and system for determining a measure of quality for images by using multi-level decomposition are presented. Multi-level decomposition of images is performed in the wavelet domain producing subbands at each level of decomposition. Aggregation of subbands is performed across multiple levels to produce an accurate measure of image quality. By aggregating only selected subbands the computational complexity of the method is greatly reduced.

RELATED APPLICATIONS

This application is a Continuation-in-Part (CIP) of U.S. applicationSer. No. 12/499,928 filed on Jul. 9, 2009, now U.S. Pat. No. 8,326,046which claims priority from U.S. provisional application 61/151,784 filedon Feb. 11, 2009. This application also claims priority from the U.S.provisional application Ser. No. 61/304,274 filed on Feb. 12, 2010.

FIELD OF THE INVENTION

The present invention relates to the assessment of image quality, and inparticular, to a method and system for determining an improved measureof quality or metric for images with enhanced perceptual performance.

BACKGROUND OF THE INVENTION

Assessment of quality of images is important in various contextsincluding image transcoding and compression. Transcoding of images iscrucial in the dissemination of rich multimedia content comprising text,voice, still and animated graphics, photos, video clips, inheterogeneous networks composed of mobile terminals, cell phones,computers and other electronic devices. Image quality can be assessed bymeasuring similarity between an original image (often referred to as areference image) and an image obtained after image processing (oftenreferred to as a distorted image). Such an assessment of quality can beused for example to determine the effectiveness of an image processingtechnique.

Image quality assessment has an important role in developing variousimage and video processing applications including multimediaapplications to provide acceptable outputs for the end-user, the humanclients. Image quality is best evaluated subjectively by human viewers.However, subjective assessment of quality is time consuming, expensive,and cannot be done for real time applications. Thus, it is necessary todefine an objective criterion that can measure the difference betweenthe undistorted original image and the distorted image signals. Ideally,such an objective measure should correlate well with the perceiveddifference between two image signals and can vary linearly with thesubjective quality. Subjective image quality is concerned with howprocessed images are perceived by a human viewer and designates hisopinion of quality.

Objective methods are usually classified based on the availability ofthe reference images. If the reference image is available, the measureof quality or the quality metric is considered as a full-reference (FR)assessment measure. The peak signal-to-noise ratio (PSNR) is the oldestand most widely used FR assessment measure and has a number ofattractive characteristics. It is simple, has clear physical meaning, isparameter free, and performs superbly in various optimization contextsas described by Z. Wang, and A. C. Bovik, in Mean squared error: Love itor leave it? A new look at signal fidelity measures, IEEE SignalProcessing Mag., vol. 26, no. 1, pp. 98-117, January 2009. This measureis defined as:

${{PSNR}\left( {X,Y} \right)} = {10 \cdot {\log_{10}\left( \frac{X_{\max}^{2}}{{MSE}\left( {X,Y} \right)} \right)}}$${{MSE}\left( {X,Y} \right)} = {\frac{1}{N_{p}} \cdot {\sum\limits_{mn}\left( {{X\left( {m,n} \right)} - {Y\left( {m,n} \right)}} \right)^{2}}}$where X and Y denote the reference (original) and distorted imagesrespectively, X_(max) is the maximum possible pixel value of thereference image X (the minimum pixel value is assumed to be zero), andN_(P) is the number of pixels in each of the images.

The conventional PSNR, however, cannot sufficiently reflect the humanperception of image fidelity, that is, a large PSNR gain may result in asmall improvement in visual quality. Thus, a number of other qualitymeasures have been developed by researches. Generally speaking, the FRassessment of image signals involves two types of approaches: abottom-up approach and a top-down approach that are discussed in Z.Wang, and A. C. Bovik, in “Modern Image Quality Assessment”, USA: Morgan& Claypool, 2006.

In the bottom-up approach, the perceptual measures of quality are bestestimated by quantifying the visibility of errors. In order to quantizeerrors according to human visual system (HVS) features, techniques inthis category try to model the functional properties of different stagesof the HVS as characterized by both psychophysical and physiologicalexperiments. This is usually accomplished in several stages ofpreprocessing, frequency analysis, contrast sensitivity analysis,luminance masking, contrast masking, and error pooling as described byZ. Wang, and A. C. Bovik, in “Modern Image Quality Assessment”, USA:Morgan & Claypool, 2006 and by A. C. Bovik, in “The Essential Guide toImage Processing”, USA: Academic Press, 2009, ch. 21. Most of HVS-basedquality assessment techniques use multi-channel models which assume thateach band of spatial frequencies is handled by an independent channel.With the visible difference predictor (VDP) model, discussed by S. Daly,in “The visible difference predictor: An algorithm for the assessment ofimage fidelity”, Proc. SPIE, vol. 1616, February 1992, pp. 2-15, theimage is decomposed into five spatial levels followed by six orientationlevels using a variation of Watson's Cortex transform. Then, a thresholdmap is computed from the contrast in that channel. In Lubin's model,which is also known as the Sarnoff visual discrimination model (VDM)presented in “A visual discrimination mode for image system design andevaluation”, Visual Models for Target Detection and Recognition,Singapore: World Scientific, 1995, pp. 207-220, the images aredecomposed into seven resolutions after low-pass filtering andresampling. P. C. Teo and D. J. Heeger uses the steerable pyramidtransform to decompose the image into several spatial frequency levelswithin which each level is further divided into a set of (six)orientation bands. Their approach is described in “Perceptual Imagedistortion”, in Proc. IEEE. Int. Conf. Image Processing, vol. 2,November 1994, pp. 982-986. VSNR, discussed by D. M. Chandler, and S. S.Hemami, in “A wavelet-based visual signal-to-noise ratio for naturalimages”, IEEE Transactions on Image Processing, vol. 16, no. 9, pp.2284-2298, September 2007, is another advanced HVS-based metric thatafter image preprocessing decomposes both the reference image and errorsbetween the reference and distorted images into five levels by using adiscrete wavelet transform and 9/7 biorthogonal filters. Then, itcomputes the contrast detection threshold to assess the ability todetect the distortions for each subband produced by the waveletdecomposition. Other known methods based on the bottom-up approach whichexploit Fourier transform rather than multiresolution decompositioninclude Weighted Signal to Noise Ratio (WSNR), discussed by N.Damera-Venkata, T. D. Kite, W. S. Geisler, B. L. Evans, and A. C. Bovik,in “Image quality assessment based on a degradation model”, IEEETransactions on Image Processing, vol. 9, no. 4, pp. 636-650, April 2000and Picture Quality Scale (PQS) described by M. Miyahara, K. Kotani, andV. R. Algazi, in “Objective Picture Quality Scale (PQS) for imagecoding”, IEEE Transactions on Communication, vol. 46, no. 9, pp.1215-1225, September 1998. Methods based on the bottom-up approach haveseveral important limitations, which are discussed by Z. Wang, and A. C.Bovik, in “Modern Image Quality Assessment”, USA: Morgan & Claypool,2006 and by Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, in “Imagequality assessment: From error visibility to structural similarity”,IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612,April 2004. Moreover, the error-based techniques, such as WSNR, NoiseQuality Measure (NQM), described by N. Damera-Venkata, T. D. Kite, W. S.Geisler, B. L. Evans, and A. C. Bovik, in “Image quality assessmentbased on a degradation model”, IEEE Transactions on Image Processing,vol. 9, no. 4, pp. 636-650, April 2000 and VSNR discussed by D. M.Chandler, and S. S. Hemami in “VSNR: A wavelet-based visualsignal-to-noise ratio for natural images”, IEEE Transactions on ImageProcessing, vol. 16, no. 9, pp. 2284-2298, September 2007 are lesssimple to use, as they require sophisticated procedures to compute thehuman visual system (HVS) parameters.

With the techniques based on the top-down approach, the overallfunctionality of the HVS is considered as a black box, and theinput/output relationship is the focus of attention. Thus, techniquesfollowing the top-down approach do not require any calibrationparameters from the HVS or viewing configuration. Two main strategies inthis category use a structural approach and an information-theoreticapproach.

The most important method using the structural approach is theStructural Similarity (SSIM) index described by Z. Wang, A. Bovik, H.Sheikh, and E. Simoncelli, in “Image quality assessment: from errorvisibility to structural similarity”, IEEE Transactions on ImageProcessing, vol. 13, no. 4, pp. 600-612, April 2004. As discussed by H.R. Sheikh, M. F. Sabir, and A. C. Bovik, in “A statistical evaluation ofrecent full reference image quality assessment algorithms”, IEEETransactions on Image Processing, vol. 15, no. 11, pp. 3440-3451,November 2006, SSIM gives an accurate score with acceptablecomputational complexity compared to other measures of quality. SSIM hasattracted a great deal of attention in recent years, and has beenconsidered for a range of applications. As described by Z. Wang, A.Bovik, H. Sheikh, and E. Simoncelli, in “Image quality assessment: fromerror visibility to structural similarity”, IEEE Transactions on ImageProcessing, vol. 13, no. 4, pp. 600-612, April 2004, the principal ideaunderlying the SSIM approach is that the HVS is highly adapted toextract structural information from visual scenes, and, therefore, ameasurement of structural similarity (or distortion) should provide agood approximation of perceptual image quality. Some approaches havetried to improve the SSIM index. The multi-scale SSIM discussed by Z.Wang, E. P. Simoncelli, and A. C. Bovik, in “Multi-scale structuralsimilarity for image quality assessment”, Proc. IEEE Asilomar Conf.Signals, Systems, Computers, vol. 2, November 2003, pp. 1398-1402,attempts to increase SSIM assessment accuracy by incorporating imagedetails at five different resolutions by applying successive low-passfiltering and downsampling. In “Understanding and simplifying thestructural similarity metric”, Proc. IEEE International Conference onImage Processing, San Diego, October 2008, pp. 1188-1191, D. M. Rouse,and S. S. Hemami investing” Discrete wavelet transform-based structuralsimilarity for image quality assessment”, Proc. IEEE InternationalConference on Image Processing, San Diego, October 2008, pp. 377-380,propose to compute it in the discrete wavelet domain using subbands atdifferent levels. Five-level decomposition using the Daubechies 9/7wavelet is applied to both original and distorted images, and then SSIMis computed between corresponding subbands. Finally, the similaritymeasure is obtained by computing a weighted mean of all SSIMs. Todetermine the weights, a large number of experiments have been performedfor measuring the sensitivity of the human eye to different frequencybands. Z. Wang, E. P. Simoncelli, in “Translation insensitive imagesimilarity in complex wavelet domain”, Proc. IEEE InternationalConference on Acoustics, Speech, Signal Processing, vol. 2, March 2005,pp. 573-576 and M. P. Sampat, Z. Wang, S. Gupta, A. C. Bovik, and M. K.Markey, in “Complex wavelet structural similarity: A new imagesimilarity index”, IEEE Transactions on Image Processing, vol. 18, no.11, pp. 2385-2401, November 2009 discuss Complex Wavelet StructuralSimilarity (CW-SSIM), which benefits from a complex version of a6-scale, 16-orientation steerable pyramid decomposition characteristicand propose a measure resistant to small geometrical distortions.

With the information-theoretic approach, visual quality assessment isviewed as an information fidelity problem. An information fidelitycriterion (IFC) for image quality measurement that is based on naturalscene statistics is presented by H. R. Sheikh, A. C. Bovik, and G. deVeciana, in “An information fidelity criterion for image qualityassessment using natural scene statistics”, IEEE Transactions on ImageProcessing, vol. 14, no. 12, pp. 2117-2128, December 2005. In the IFC,the image source is modeled by using a Gaussian scale mixture (GSM)while the image distortion process is modeled as an error-pronecommunication channel. The information shared between the images beingcompared is quantified by using the mutual information that is astatistical measure of information fidelity. Anotherinformation-theoretic quality metric is the “Visual Information Fidelity(VIF) index” discussed by H. R. Sheikh, and A. C. Bovik, in “Imageinformation and visual quality”, IEEE Transactions on Image Processing,vol. 15, no. 2, pp. 430-444, February 2006. The computation of the VIFindex follows the same procedure as the IFC, except that, in thedetermination of the VIF index both the image distortion process and thevisual perception process are modeled as error-prone communicationchannels. For the VIF measure, the HVS distortion channel is modeledwith an additive white Gaussian noise. The VIF index is the mostaccurate image measure of quality according to the performanceevaluation of prominent image quality assessment algorithms presented byH. R. Sheikh, M. F. Sabir, and A. C. Bovik, in “A statistical evaluationof recent full reference image quality assessment algorithms”, IEEETransactions on Image Processing, vol. 15, no. 11, pp. 3440-3451,November 2006.

Review of existing literature reveals a number of shortcomings of theprior art methods. The limitations of these prior art methods includethe following.

First, computational complexity of the existing assessment techniquesfor accurately determining the measures of quality are very high. Someimage/video processing applications, like identifying the bestquantization parameters for each frame in video encoding described by S.L. P. Yasakethu, W. A. C. Fernando, S. Adedoyin, and A. Kondoz in apaper entitled “A rate control technique for off line H.264/AVC videocoding using subjective quality of video”, IEEE Transactions on ConsumerElectronics, vol. 54, no. 3, pp. 1465-1472, August 2008, could be usedmore efficiently if an accurate low-complexity technique for determiningthe measure of quality (quality metric) were used.

Second, the bottom-up approach used by prior art methods requires thatthe associated techniques apply a multiresolution transform, decomposethe input image into a large number of resolutions (five or more).While, the HVS is a complex system which is not completely known to us,combining the different bands into a final metric is difficult. Insimilar top-down methods such as multi-scale and multi-level SSIMsdiscussed by Z. Wang, E. P. Simoncelli, and A. C. Bovik, in “Multi-scalestructural similarity for image quality assessment”, Proc. IEEE AsilomarConf. Signals, Systems, Computers, vol. 2, November 2003, pp. 1398-1402and by C.-L. Yang, W.-R. Gao, cited earlier and L.-M. Po, in “Discretewavelet transform-based structural similarity for image qualityassessment”, Proc. IEEE Int. Conf. Image Processing, San Diego, October2008, pp. 377-380, cited earlier, determining the sensitivity of the HVSto different scales or subbands requires many experiments. Moreover, ifthe wavelet or filter is changed, the computed weights and parametersare no longer optimum and may not even be valid.

Third, top-down methods, such as SSIM, gather local statistics within asquare sliding window and may not always be very accurate.

Fourth, the large number of decomposition levels, as discussed by C.-L.Yang, W.-R. Gao, and L.-M. Po, in “Discrete wavelet transform-basedstructural similarity for image quality assessment”, Proc. IEEE Int.Conf. Image Processing, San Diego, October 2008, pp. 377-380, cited inthe previous paragraph would make the size of the approximation subbandthat has the main image contents very small, and it would no longer beable to help in the effective extraction of image statistics.

Fifth, previous SSIM methods use the mean of the SSIM quality map todetermine the measure of quality for the image (or the overall imagequality score). However, distortions in various image areas havedifferent impacts on the HVS.

Thus, there is a further need for the development of an improved measureof quality for images, which would avoid or mitigate the disadvantagesof the prior art.

SUMMARY OF THE INVENTION

Therefore it is an objective of the present invention to provide animproved method and system for determining measures of image qualitythat mitigate the shortcomings of the prior art.

According to one aspect of the invention, there is provided a method fordetermining a measure of quality for a distorted image Y, characterizinga similarity between the image Y and an undistorted reference image X,having the same number of rows and columns of pixels as the image Y, themethod comprising:

(a1) applying a N level multiresolution decomposition, comprising levels1, 2, . . . i, i+1, . . . N, to the image X, to produce:

-   -   for each level i, with i ranging from 1 to N−1, intermediate        subbands of image X for processing at level i+1; and    -   for the level N, an approximation subband containing main        content of the image X and detail subbands containing edges of        the image X;        (b1) applying said N level multiresolution decomposition,        comprising levels 1, 2, . . . i, i+1, . . . N, to the image Y,        to produce:    -   for each level i, with i ranging from 1 to N−1, intermediate        subbands of image Y for processing at level i+1; and    -   for the level, N, an approximation subband containing main        content of the image Y and detail subbands containing edges of        the image Y;        (c1) applying an image quality metric (IQM) to the approximation        subband of the image X and the approximation subband of the        image Y to produce an approximation quality measure        characterizing similarity between the main content of the image        X and the main content of the image Y;        (d1) aggregating the intermediate subbands at the level i for        the image X, with i ranging from levels 1 to N−1, and the detail        subbands of the image X to produce an edge map of the image X        characterizing the edges of the image X;        (e1) aggregating the intermediate subbands at the level i for        the image Y, with i ranging from 1 to N−1, and the detail        subbands of the image Y to produce an edge map of the image Y        characterizing the edges of image Y;        (f1) applying the IQM between the edge map of the image X and        the edge map of the image Y to produce an edge quality measure        characterizing similarity between the edges of the image X and        the edges of the image Y; and        (g1) processing the approximation quality measure and the edge        quality measure to determine the measure of quality.        In the method described above:        (a2) the step (d1) further comprises:    -   (a2i) selecting the intermediate subbands at each level i, with        i ranging from 1 to N−1, and the detail subbands for the image X        based on an accuracy to be achieved in determining the measure        of quality; and    -   (a2ii) aggregating only selected intermediate subbands and        selected detail subbands for the image X; and        (b2) the step (e1) further comprises:    -   (b2i) selecting the intermediate subbands at said each level i        and the detail subbands for the image Y based on said accuracy        to be achieved in determining the measure of quality; and    -   (b2ii) aggregating only selected intermediate subbands and        selected detail subbands for the image Y.

The step (a2i) and the step (b2i) further comprise selecting theintermediate subbands at said each level i and the detail subbands forthe image X and the image Y based on a number of the intermediate andthe detailed subbands required for achieving the accuracy.

The method further comprises:

(a4) at the step (a2ii):

-   -   (a4i) for said each level i, aggregating the selected        intermediate subbands for the image X producing an edge map at        the level i for the image X;    -   (a4ii) aggregating the selected detail subbands for the image X        producing an edge map at the level N for the image X; and    -   (a4iii) aggregating the edge map at said each level i for the        image X with the edge map at the level N for the image X; and        (b4) at the step (b2ii):    -   (b4i) for said each level i, aggregating the selected        intermediate subbands for the image Y producing an edge map at        the level i for the image Y;    -   (b4ii) aggregating the selected detail subbands for the image Y        producing an edge map at the level N for the image Y; and    -   (b4iii) aggregating the edge map at said each level i for the        image Y with the edge map at the level N for the image Y.

In the method described above:

(a5) the step (a4iii) further comprises:

-   -   (a5i) multiplying the edge map at said each level i for the        image X with a predetermined weight for the level i;    -   (a5ii) multiplying the edge map at the level N for the image X        with a predetermined weight at the level N; and    -   (a5iii) summing products resulting from the multiplying in the        step (a5i) and a product resulting from the multiplying in the        step (a5ii); and        (b5) the step (b4iii) further comprises:    -   (b5i) multiplying the edge map at said each level i for the        image Y with the predetermined weight for the level i;    -   (b5ii) multiplying the edge map at the level N for the image Y        with the predetermined weight at the level N; and    -   (b5iii) summing products resulting from the multiplying in the        step (b5i) and a product resulting from the multiplying in the        step (b5ii).

In the method described above:

(a6) the step (a4i) comprises:

-   -   (a6i) for said each level i, squaring each selected intermediate        subband for the image X;    -   (a6ii) multiplying a result of the squaring performed in the        step (a6i) with a predetermined weight for said each selected        intermediate subband for the image X;    -   (a6iii) summing products resulting from the multiplying        performed in the step (a6ii);    -   (a6iv) applying a square root function to a result of the        summing performed in the step (a6iii) producing the edge map at        the level i for the image X;        (b6) the step (a4ii) comprises:    -   (b6i) squaring each said selected detail subband for the image        X;    -   (b6ii) multiplying a result of the squaring performed in the        step (b6i) with a predetermined weight for said each selected        detail subband for the image X;    -   (b6iii) summing products resulting from the multiplying        performed in the step (b6ii);    -   (b6iv) applying a square root function to a result of the        summing performed in the step (b6iii) producing the edge map at        the level N for the image X;        (c6) the step (b4i) comprises:    -   (c6i) for said each level i, squaring each selected intermediate        subband for the image Y;    -   (c6ii) multiplying a result of the squaring performed in the        step (c6i) with a predetermined weight for said each selected        intermediate subband for the image Y;    -   (c6iii) summing products resulting from the multiplying        performed in the step (c6ii);    -   (c6iv) applying a square root function to a result of the        summing performed in the step (c6iii) producing the edge map at        the level i for the image Y; and        (d6) the step (b4ii) comprises:    -   (d6i) squaring each said selected detail subband for the image        Y;    -   (d6ii) multiplying a result of the squaring performed in the        step (d6i) with a predetermined weight for said each selected        detail subband for the image Y;    -   (d6iii) summing products resulting from the multiplying        performed in the step (d6ii);    -   (d6iv) applying a square root function to a result of the        summing performed in the step (d6iii) producing the edge map at        the level N for the image Y.

The intermediate subbands at said each level i for the image X and theimage Y include for each respective image:

-   -   level i-detail subbands and level i-wavelet packet (WP)        subbands;    -   the level i-WP subbands comprising level i-WP approximation        subbands and level i-WP detail subbands; and    -   the level i-WP approximation subbands further comprising a level        i-horizontal WP approximation subband, a level i-vertical WP        approximation subband and a level i-diagonal WP approximation        subband.

In the method described above:

(a8) the step (a2i) further comprises selecting one or more of the leveli-WP approximation subbands and one or more of the detail subbands forthe image X; and

(b8) the step (b2i) further comprises selecting one or more of the leveli-WP approximation subbands and one or more of the detail subbands forthe image Y.

In the method described above:

(a9) the step (a8) further comprises including one or more of the leveli-detail subbands for the image X in the selecting; and

(b9) the step (b8) further comprises including one or more of the leveli-detail subbands for the image Y in the selecting.

In the method described above:

(a10) the step (a2ii) further comprises:

-   -   (a10i) aggregating the level i-horizontal WP approximation        subband for the image X at said each level i and the horizontal        subband for the image X producing a horizontal edge map for the        image X;    -   (a10ii) aggregating the level i-vertical WP approximation        subband for the image X at said each level i and the vertical        subband for the image X producing a vertical edge map for the        image X;    -   (a10iii) aggregating the level i-diagonal WP approximation        subband for the image X at said each level i and the diagonal        subband for the image X producing a diagonal edge map for the        image X; and    -   (a10iv) aggregating the horizontal edge map, the vertical edge        map and the diagonal edge map for the image X; and        (b10) the step (b2ii) further comprises:    -   (b10i) aggregating the level i-horizontal WP approximation        subband for the image Y at said each level i and the horizontal        subband for the image Y producing a horizontal edge map for the        image Y;    -   (b10ii) aggregating the level i-vertical WP approximation        subband for the image Y at said each level i and the vertical        subband for the image Y producing a vertical edge map for the        image Y;    -   (b10iii) aggregating the level i-diagonal WP approximation        subband for the image Y at said each level i and the diagonal        subband for the image Y producing a diagonal edge map for the        image Y; and    -   (b10iv) aggregating the horizontal edge map, the vertical edge        map and the diagonal edge map for the image Y.

In the method described above, the level i-WP detail subbands for theimage X and the image Y further comprise for each respective image, alevel i-WP horizontal subband, a level i-WP vertical subband and a leveli-WP diagonal subband, and:

(a11) the step (a10i) further comprises including the level i-WPhorizontal subband for the image X in the aggregating;

(b11) the step (a10ii) further comprises including the level i-WPvertical subband for the image X in the aggregating;

(c11) the step (a10iii) further comprises including the level i-WPdiagonal subband for the image X in the aggregating;

(d11) the step (b10i) further comprises including the level i-WPhorizontal subband for the image Y in the aggregating;

(e11) the step (b10ii) further comprises including the level i-WPvertical subband for the image Y in the aggregating; and

(f11) the step (b10iii) further comprises including the level i-WPdiagonal subband for the image Y in the aggregating.

In the method described above:

(a12) the step (a10iv) comprises:

-   -   (a12i) applying a square root function to the horizontal edge        map, the vertical edge map and the diagonal edge map for the        image X; and    -   (a12ii) summing results of applying said square root function;        and        (b12) the step (b10iv) comprises:    -   (b12i) applying a square root function to the horizontal edge        map, the vertical edge map and the diagonal edge map for the        image Y; and    -   (b12ii) summing results of applying said square root function.

In the method described above:

(a13) the step (a10iv) comprises:

-   -   (a13i) summing the horizontal edge map, the vertical edge map        and the diagonal edge map for the image X; and    -   (a13ii) applying a square root function to result of the summing        performed in the step (a13i); and        (b13) the step (b10iv) comprises:    -   (b13i) summing the horizontal edge map, the vertical edge map        and the diagonal edge map for the image Y; and    -   (b13ii) applying a square root function to result of the summing        performed in the step (b13i).

In the method described above, the steps (a1) and (b1) comprise applyinga N level discrete wavelet transform (DWT). Alternatively, the discretewavelet transform may be a Haar transform, or one of a Newlandtransform, or wavelet transform using a Daubechies filter.

The method described above comprises one or more of the following:

(a17) at the step (a1):

-   -   (a17i) for the level 1, applying the multiresolution        decomposition to the image X producing intermediate subbands at        the level 1 for processing at level 2; and    -   (a17ii) for the level i, with i ranging from 2 to N, applying        the multiresolution decomposition to one or more of the        intermediate subbands produced by the multiresolution        decomposition performed at the level i−1; and        (b17) at the step (b1):    -   (b17i) for the level 1, applying the multiresolution        decomposition to the Image Y producing intermediate subbands at        the level 1 for processing at level 2; and    -   (b17ii) for the level i, with i ranging from 2 to N, applying        the multiresolution decomposition to one or more of the        intermediate subbands produced by the multiresolution        decomposition performed at the level i−1.

In the embodiments of the invention, the approximation quality measureis an approximation quality map, and the edge quality measure is an edgequality map, and the step (g1) further comprises:

(a18) generating a contrast map, including assigning correspondingvalues to the pixels of the approximation subband and the edge map ofthe image X and the image Y according to their respective importance tohuman visual system;

(b18) performing weighted pooling of the approximation quality map usingthe contrast map to produce an approximation quality score;

(c18) performing weighted pooling of the edge quality map using thecontrast map to produce an edge quality score; and

(d18) combining the approximation similarity score with the edgesimilarity score from the step (c18) to determine the measure ofquality.

In the embodiments of the invention, the method further comprisesdetermining N as a function of a minimum size of the approximationsubband, S, which produces a substantially peak response for humanvisual system.

In the method described above:

(a20) the step (c1) comprises applying a structural similarity (SSIM)IQM to the approximation subband of the image X and the approximationsubband of the image Y to produce an approximation SSIM map, SSIM_(A);

(b20) the step (f1) comprises applying the SSIM IQM between the edge mapof the image X and the edge map of the image Y to produce an edge SSIMmap, SSIM_(E); and

(c20) the step (g1) comprises processing the SSIM_(A) and the SSIM_(E)to determine a SSIM_(DWT) score as the measure of quality.

Alternatively:

(a21) the step (c1) comprises applying an Absolute Difference (AD) IQMto the approximation subband of the image X and the approximationsubband of the image Y to produce an approximation AD map, AD_(A);

(b21) the step (f1) comprises applying the AD IQM between the edge mapof the image X and the edge map of the image Y to produce an edge ADmap, AD_(E); and

(c21) the step (g1) comprises processing the AD_(A) and the AD_(E) todetermine an AD_(DWT) score as the measure of quality.

Yet alternatively:

(a22) the step (c1) comprises applying a peak-signal-to-noise ratio(PSNR) IQM to the approximation subband of the image X and theapproximation subband of the image Y to produce a PSNR approximationquality score, PSNR_(A);

(b22) the step (f1) comprises applying the PSNR IQM between the edge mapof the image X and the edge map of the image Y to produce a PSNR edgequality score, PSNR_(E); and

(c22) the step (g1) comprises processing the PSNR_(A) and the PSNR_(E)to determine a PSNR_(DWT) score as the measure of quality.

Yet alternatively:

(a23) the step (c1) comprises applying a Visual Information Fidelity(VIF) IQM to the approximation subband of the image X and theapproximation subband of the image Y to produce a VIF approximationquality score, VIF_(A);

(b23) the step (f1) comprises applying the VIF IQM between the edge mapof the image X and the edge map of the image Y to produce a VIF edgequality score, VIF_(E); and

(c23) the step (g1) comprises processing the VIF_(A) and the VIF_(E) todetermine a VIF_(DWT) score as the measure of quality.

In the methods described above:

(a24) the step (a2) further comprises choosing the selected intermediatesubbands at said each level i and the detail subbands for the image X,having substantially same resolution, for aggregating; and

(b24) the step (b2) further comprises choosing the selected intermediatesubbands at said each level i and the detail subbands for the image Y,having substantially same resolution, for aggregating.

According to another aspect of the invention, there is provided a systemfor determining a measure of quality for a distorted image Y,characterizing a similarity between the image Y and an undistortedreference image X, having the same number of rows and columns of pixelsas the image Y, the system comprising:

a processor, and a computer readable storage medium having computerreadable instructions stored thereon, which, when executed by theprocessor, form the following:

-   -   (a25) a first decomposition unit applying a N level        multiresolution decomposition, comprising levels 1, 2, . . . i,        i+1, . . . N, to the image X, to produce:        -   for each level i, with i ranging from 1 to N−1, intermediate            subbands of image X for processing at level i+1; and        -   for the level N, an approximation subband containing main            content of the image X and detail subbands containing edges            of the image X;            (b25) a second decomposition unit applying said N level            multiresolution decomposition, comprising levels 1, 2, . . .            i, i+1, . . . N, to the image Y, to produce:    -   for each level i, with i ranging from 1 to N−1, intermediate        subbands for image Y; and    -   for the level, N, an approximation subband containing main        content of the image Y and detail subbands containing edges of        the image Y;        (c25) an approximation quality measure unit applying an image        quality metric (IQM) to the approximation subband of the image X        and the approximation subband of the image Y to produce an        approximation quality measure characterizing similarity between        the main content of the image X and the main content of the        image Y;        (d25) a first aggregation unit aggregating the intermediate        subbands at the level i for the image X, with i ranging from        levels 1 to N−1 and the detail subbands of the image X to        produce an edge map of the image X characterizing the edges of        the image X;        (e25) a second aggregation unit aggregating the intermediate        subbands at the level i for the image Y, with i ranging from 1        to N−1 and the detail subbands of the image Y to produce an edge        map of the image Y characterizing the edges of image Y;        (f25) an edge quality measure unit applying the IQM between the        edge map of the image X and the edge map of the image Y to        produce an edge quality measure characterizing similarity        between the edges of the image X and the edges of the image Y;        and        (g25) a quality measure unit processing the approximation        quality measure and the edge quality measure to determine the        measure of quality.

In the system described above:

(a26) the first aggregation unit (d25) further comprises:

-   -   (a26i) a first selection unit selecting the intermediate        subbands at each level i, with i ranging from 1 to N−1, and the        detail subbands for the image X based on an accuracy to be        achieved in determining the measure of quality; and    -   (a26ii) a first selective aggregation unit aggregating only        selected intermediate subbands and selected detail subbands for        the image X; and        (b26) the second aggregation unit (e1) further comprises:    -   (b26i) a second selection unit selecting the intermediate        subbands at said each level i and the detail subbands for the        image Y based on said accuracy to be achieved in determining the        measure of quality; and    -   (b26ii) a second selective aggregation unit aggregating only        selected intermediate subbands and selected detail subbands for        the image Y.

In the system described above:

(a27) the first selection unit (a26i) further comprises a firstselection sub-unit selecting the intermediate subbands at said eachlevel i and the detail subbands for the image X based on a number of theintermediate and the detailed subbands required for achieving theaccuracy; and(b27) the second selection unit (b26i) further comprises a secondselection sub-unit selecting the intermediate subbands at said eachlevel i and the detail subbands for the image Y based on a number of theintermediate and the detailed subbands required for achieving theaccuracy.

In the system described above:

(a28) at the first selective aggregation unit (a26ii):

-   -   (a28i) a first level i-intermediate subbands aggregation unit        aggregating, for said each level i, the selected intermediate        subbands for the image X producing an edge map at the level i        for the image X;    -   (a28ii) a first detail subbands aggregation unit aggregating the        selected detail subbands for the image X producing an edge map        at the level N for the image X; and    -   (a28iii) a first edge map determination unit aggregating the        edge map at said each level i for the image X with the edge map        at the level N for the image X; and        (b28) at the second selective aggregation unit (b26ii):    -   (b28i) a second level i-intermediate subbands aggregation unit        aggregating, for said each level i, the selected intermediate        subbands for the image Y producing an edge map at the level i        for the image Y;    -   (b28ii) a second detail subbands aggregation unit aggregating        the selected detail subbands for the image Y producing an edge        map at the level N for the image Y; and    -   (b28iii) a second edge map determination unit aggregating the        edge map at said each level i for the image Y with the edge map        at the level N for the image Y.

In the system described above:

(a29) the first edge map determination unit (a28iii) further comprises afirst computation unit for:

-   -   multiplying the edge map at said each level i for the image X        with a predetermined weight for the level i;    -   multiplying the edge map at the level N for the image X with a        predetermined weight for the level N; and    -   summing products resulting from the multiplying the edge map at        said each level i for the image X and the multiplying the edge        map at the level N for the image X.        (b29) the second edge map determination unit (b28iii) further        comprises a second computation unit for:    -   multiplying the edge map at said each level i for the image Y        with the predetermined weight for the level i;    -   multiplying the edge map at the level N for the image Y with the        predetermined weight for the level N; and    -   summing products resulting from the multiplying the edge map at        said each level i for the image Y and the multiplying the edge        map at the level N for the image Y.

In the system described above:

(a30) the first level i-intermediate subbands aggregation unit (a28i)comprises a first level i-computation unit for:

-   -   squaring each selected intermediate subband for the image X for        said each level i; multiplying a result of the squaring with a        predetermined weight for said each selected intermediate subband        for the image X;    -   summing products resulting from the multiplying; and    -   applying a square root function to a result of the summing        producing the edge map at the level i for the image X;        (b30) the first detail subbands aggregation unit (a28ii)        comprises a first detail subbands computation unit for:    -   squaring each said selected detail subband for the image X;    -   multiplying a result of the squaring with a predetermined weight        for said each selected detail subband for the image X;    -   summing products resulting from the multiplying; and    -   applying a square root function to a result of the summing        producing the edge map at the level N for the image X;        (c30) the second level i-intermediate subbands aggregation unit        (b28i) comprises a second level i-computation unit for:    -   squaring each said selected intermediate subband for the image Y        for said each level i;    -   multiplying a result of the squaring performed with a        predetermined weight for said each selected intermediate subband        for the image Y;    -   summing products resulting from the multiplying;    -   applying a square root function to a result of the summing        producing the edge map at the level i for the image Y; and        (d30) the second detail subbands aggregation unit (b28ii)        comprises a second detail subbands computation unit for:    -   squaring each said selected detail subband for the image Y;    -   multiplying a result of the squaring with a predetermined weight        for said each selected detail subband for the image Y;    -   summing products resulting from the multiplying; and    -   applying a square root function to a result of the summing        producing the edge map at the level N for the image Y.

In the system described above, the intermediate subbands at said eachlevel i for the image X and the image Y include for each respectiveimage:

-   -   level i-detail subbands and level i-wavelet packet (WP)        subbands;    -   the level i-WP subbands comprising level i-WP approximation        subbands and level i-WP detail subbands; and    -   the level-WP approximation subbands further comprising a level        i-horizontal WP approximation subband, a level i-vertical WP        approximation subband and a level i-diagonal WP approximation        subband.

In the system described above:

(a32) the first selection unit (a26i) further comprises a firstWP-detail selection sub-unit selecting one or more of the level i-WPapproximation subbands and one or more of the detail subbands for theimage X; and

(b32) the second selection unit (b26i) further comprises a secondWP-detail selection sub-unit selecting one or more of the level i-WPapproximation subbands and one or more of the detail subbands for theimage Y.

In the system described above:

(a33) the first WP-detail selection sub-unit (a32) further comprises afirst detail-selection module including one or more of the leveli-detail subbands for the image X in the selecting; and

(b33) the second WP-detail selection sub-unit (b32) further comprises asecond detail-selection module including one or more of the leveli-detail subbands for the image Y in the selecting.

In the system described above:

(a34) the first selective aggregation unit (a26ii) further comprises:

-   -   (a34i) a first horizontal edge map unit aggregating the level        i-horizontal WP approximation subband for the image X at said        each level i and the horizontal subband for the image X        producing a horizontal edge map for the image X;    -   (a34ii) a first vertical edge map unit aggregating the level        i-vertical WP approximation subband for the image X at said each        level i and the vertical subband for the image X producing a        vertical edge map for the image X;    -   (a34iii) a first diagonal edge map unit aggregating the level        i-diagonal WP approximation subband for the image X at said each        level i and the diagonal subband for the image X producing a        diagonal edge map for the image X; and    -   (a34iv) a first combination unit aggregating the horizontal edge        map, the vertical edge map and the diagonal edge map for the        image X; and        (b34) the second selective aggregation unit (b26ii) further        comprises:    -   (b34i) a second horizontal edge map unit aggregating the level        i-horizontal WP approximation subband for the image Y at said        each level i and the horizontal subband for the image Y        producing a horizontal edge map for the image Y;    -   (b34ii) a second vertical edge map unit aggregating the level        i-vertical WP approximation subband for the image Y at said each        level i and the vertical subband for the image Y producing a        vertical edge map for the image Y;    -   (b34iii) a second diagonal edge map unit aggregating the level        i-diagonal WP approximation subband for the image Y at said each        level i and the diagonal subband for the image Y producing a        diagonal edge map for the image Y; and    -   (b34iv) a second combination unit aggregating the horizontal        edge map, the vertical edge map and the diagonal edge map for        the image Y.

In the system described above, the level i-WP detail subbands for theimage X and the image Y further comprise, for each respective image, alevel i-WP horizontal subband, a level i-WP vertical subband and a leveli-WP diagonal subband; and

(a35) the first horizontal edge map unit (a34i) further comprises afirst horizontal sub-unit including the level i-WP horizontal subbandfor the image X in the aggregating;

(b35) the first vertical edge map unit (a34ii) further comprises a firstvertical sub-unit including the level i-WP vertical subband for theimage X in the aggregating;

(c35) the first diagonal edge map unit (a34iii) further comprises afirst diagonal sub-unit including the level i-WP diagonal subband forthe image X in the aggregating;

(d35) the second horizontal edge map unit (b34i) further comprises asecond horizontal sub-unit including the level i-WP horizontal subbandfor the image Y in the aggregating;

(e35) the second vertical edge map unit (b34ii) further comprises asecond vertical sub-unit including the level i-WP vertical subband forthe image Y in the aggregating; and

(f35) the second diagonal edge map unit (b34iii) further comprises asecond diagonal sub-unit including the level i-WP diagonal subband forthe image Y in the aggregating.

In the system described above:

(a36) the first combination unit (a34iv) comprises a first combinationsub-unit for:

-   -   applying a square root function to the horizontal edge map, the        vertical edge map and the diagonal edge map for the image X; and    -   summing results of applying said square root function; and        (b36) the second combination unit (b34iv) comprises a second        combination sub-unit for:    -   applying a square root function to the horizontal edge map, the        vertical edge map and the diagonal edge map for the image Y; and    -   summing results of applying said square root function.

In the system described above:

(a37) the first combination unit (a34iv) comprises a third combinationsub-unit for:

-   -   summing the horizontal edge map, the vertical edge map and the        diagonal edge map for the image X; and    -   applying a square root function to result of the summing; and        (b37) the second combination unit (b34iv) comprises a fourth        combination sub-unit for:    -   summing the horizontal edge map, the vertical edge map and the        diagonal edge map for the image Y; and    -   applying a square root function to result of the summing.

In the system described above, the first decomposition unit (a1) and thesecond decomposition unit (b1) comprise computational means for applyinga N level discrete wavelet transform (DWT).

In the system described above, the discrete wavelet transform is a Haartransform. Alternatively, the discrete wavelet transform is one of aNewland transform, or wavelet transform using a Daubechies filter.

The system of the embodiments of the invention may comprise one or moreof the following:

(a41) at the first decomposition unit (a25):

-   -   (a41i) a first level 1 decomposition unit applying the        multiresolution decomposition to the image X producing        intermediate subbands at the level 1 for processing at level 2;    -   (a41ii) a first level i decomposition unit applying the        multiresolution decomposition to one or more of the intermediate        subbands produced by the multiresolution decomposition performed        at the level i−1, with i ranging from 2 to N;        (b41) at the second decomposition unit (b25):    -   (b41i) a second level 1 decomposition unit applying the        multiresolution decomposition to the Image Y producing        intermediate subbands at the level 1 for processing at level 2;    -   (b41ii) a second level i decomposition unit applying the        multiresolution decomposition to one or more of the intermediate        subbands produced by the multiresolution decomposition performed        at the level i−1, with i ranging from 2 to N.

In the system described above, the approximation quality measure is anapproximation quality map, and the edge quality measure is an edgequality map, and the quality measure unit (g25) further comprises:

(a42) a contrast map unit generating a contrast map, including assigningcorresponding values to the pixels of the approximation subband and theedge map of the image X and the image Y according to their respectiveimportance to human visual system;

(b42) a first pooling unit performing weighted pooling of theapproximation quality map using the contrast map to produce anapproximation quality score;

(c42) a second pooling unit performing weighted pooling of the edgequality map using the contrast map to produce an edge quality score; and

(d42) a score combination unit combining the approximation similarityscore from the first pooling unit (b42) with the edge similarity scorefrom the second pooling unit (c42) to determine the measure of quality.

The system as described above may further comprise computational meansfor determining N as a function of a minimum size of the approximationsubband, S, which produces a substantially peak response for humanvisual system.

In the system of the embodiments of the invention:

(a44) the approximation quality measure unit (c25) comprises anapproximation SSIM map unit applying a SSIM IQM to the approximationsubband of the image X and the approximation subband of the image Y toproduce an approximation structural similarity (SSIM) map, SSIM_(A);(b44) the edge quality measure unit (f25) comprises an edge SSIM mapunit applying the SSIM IQM between the edge map of the image X and theedge map of the image Y to produce an edge SSIM map, SSIM_(E); and(c44) the quality measure unit (g25) comprises a SSIM_(DWT) score unitprocessing the SSIM_(A) and the SSIM_(E) to determine a SSIM_(DWT) scoreas the measure of quality.

In the system described above:

(a45) the approximation quality measure unit (c25) comprises anapproximation AD map unit applying an AD IQM to the approximationsubband of the image X and the approximation subband of the image Y toproduce an approximation AD map, AD_(A);

(b45) the edge quality measure unit (f25) comprises an edge AD map unitapplying the AD IQM between the edge map of the image X and the edge mapof the image Y to produce an edge AD map, AD_(E); and

(c45) the quality measure unit (g25) comprises an AD_(DWT) score unitprocessing the AD_(A) and the AD_(E) to determine an AD_(DWT) score asthe measure of quality.

Alternatively:

(a46) the approximation quality measure unit (c25) comprises a PSNRapproximation quality score unit applying a PSNR IQM to theapproximation subband of the image X and the approximation subband ofthe image Y to produce a PSNR approximation quality score, PSNR_(A);(b46) the edge quality measure unit (f25) comprises a PSNR edge qualityscore determination unit applying the PSNR IQM between the edge map ofthe image X and the edge map of the image Y to produce a PSNR edgequality score, PSNR_(E); and(c46) the quality measure unit (g25) comprises a PSNR_(DWT) score unitprocessing the PSNR_(A) and the PSNR_(E) to determine a PSNR_(DWT) scoreas the measure of quality.

Yet alternatively:

(a47) the approximation quality measure unit (c25) comprises a VIFapproximation quality score unit applying a VIF IQM to the approximationsubband of the image X and the approximation subband of the image Y toproduce a VIF approximation quality score, VIF_(A);(b47) the edge quality measure unit (f25) comprises a VIF edge qualityscore determination unit applying the VIF IQM between the edge map ofthe image X and the edge map of the image Y to produce a VIF edgequality score, VIF_(E); and(c47) the quality measure unit (g25) comprises a VIF_(DWT) score unitprocessing the VIF_(A) and the VIF_(E) to determine a VIF_(DWT) score asthe measure of quality.

Yet alternatively:

(a48) the first aggregation unit (a26) further comprises computationalmeans for choosing the selected intermediate subbands at said each leveli and the detail subbands for the image X having substantially sameresolution for aggregating; and

(b48) the second aggregation unit (b26) further comprises computationalmeans for choosing the selected intermediate subbands at said each leveli and the detail subbands for the image Y having substantially sameresolution for aggregating.

According to yet another aspect of the invention, there is provided acomputer readable storage medium, having computer readable program codeinstructions stored thereon, which, when executed by a processor,perform the following:

(a49) applying a N level multiresolution decomposition, comprisinglevels 1, 2, . . . i, i+1, . . . N, to the image X, to produce:

-   -   for each level i, with i ranging from 1 to N−1, intermediate        subbands of image X for processing at level i+1; and    -   for the level N, an approximation subband containing main        content of the image X and detail subbands containing edges of        the image X;        (b49) applying said N level multiresolution decomposition,        comprising levels 1, 2, . . . i, i+1, . . . N, to the image Y,        to produce:    -   for each level i, with i ranging from 1 to N−1, intermediate        subbands of image Y for processing at level i+1; and    -   for the level, N, an approximation subband containing main        content of the image Y and detail subbands containing edges of        the image Y;        (c49) applying an image quality metric (IQM) to the        approximation subband of the image X and the approximation        subband of the image Y to produce an approximation quality        measure characterizing similarity between the main content of        the image X and the main content of the image Y;        (d49) aggregating the intermediate subbands at the level i for        the image X, with i ranging from levels 1 to N−1, and the detail        subbands of the image X to produce an edge map of the image X        characterizing the edges of the image X;        (e49) aggregating the intermediate subbands at the level i for        the image Y, with i ranging from 1 to N−1, and the detail        subbands of the image Y to produce an edge map of the image Y        characterizing the edges of image Y;        (f49) applying the IQM between the edge map of the image X and        the edge map of the image Y to produce an edge quality measure        characterizing similarity between the edges of the image X and        the edges of the image Y; and        (g49) processing the approximation quality measure and the edge        quality measure to determine the measure of quality.

Thus, improved methods and systems for determining a quality measure foran image using multi-level decomposition of images have been provided.

BRIEF DESCRIPTION OF THE DRAWINGS

Further features and advantages of the invention will be apparent fromthe following description of the embodiment, which is described by wayof example only and with reference to the accompanying drawings, inwhich:

FIG. 1( a) presents system 100 of the embodiment of the invention;

FIG. 1( b) illustrates the number of levels unit 102 of FIG. 1( a) inmore detail;

FIG. 1( c) illustrates the first decomposition unit 103 of FIG. 1( a) inmore detail;

FIG. 1( d) illustrates the second decomposition unit 104 of FIG. 1( a)in more detail;

FIG. 1( e) illustrates the first aggregation unit 113 of FIG. 1( a) inmore detail;

FIG. 1( f) illustrates the second aggregation unit 114 of FIG. 1( a) inmore detail;

FIG. 1( g) illustrates the approximation quality measure unit 105, thequality measure unit 108 and the edge quality measure unit 109 of FIG.1( a) in more detail for an alternate embodiment based on the SSIM IQM;

FIG. 1( h) illustrates the approximation quality measure unit 105, thequality measure unit 108 and the edge quality measure unit 109 of FIG.1( a) in more detail for an alternate embodiment based on the AD IQM;

FIG. 1( i) illustrates the approximation quality measure unit 105, thequality measure unit 108 and the edge quality measure unit 109 of FIG.1( a) in more detail for an alternate embodiment based on the PSNR IQM;

FIG. 1( j) illustrates the approximation quality measure unit 105, thequality measure unit 108 and the edge quality measure unit 109 of FIG.1( a) in more detail for an alternate embodiment based n the VIF IQM;

FIG. 1( k) presents system 100 a, an alternate embodiment of theinvention;

FIG. 1( l) illustrates the first aggregation unit 113 a of FIG. 1( k) inmore detail;

FIG. 1( m) illustrates the second aggregation unit 114 a of FIG. 1( k)in more detail;

FIG. 1( n) illustrates the first selective aggregation unit 171 of FIG.1( l) for an alternate embodiment in more detail;

FIG. 1( o) illustrates the second selective aggregation unit 183 of FIG.1( m) for an alternate embodiment in more detail;

FIG. 1( p) illustrates the second combination unit 197 of FIG. 1( n) inmore detail;

FIG. 1( q) illustrates the first combination unit 206 of FIG. 1( o) inmore detail;

FIG. 1( r) illustrates the first decomposition unit 103 a of FIG. 1( k)in more detail;

FIG. 1( s) illustrates the second decomposition unit 104 a of FIG. 1( k)in more detail;

FIG. 2 (a) presents a block diagram providing a high level descriptionof the method used by the embodiments of the invention for computing themeasure of quality;

FIG. 2 (b) presents a flowchart showing the steps of the method used inthe general framework of the system 100 for computing a measure ofquality.

FIG. 3( a 1) presents the subbands of image X for an example two-leveldecomposed image;

FIG. 3( a 2) presents the subbands of image X for an example three-leveldecomposed image;

FIG. 3( b) presents an example original image;

FIG. 3( c) presents a contrast map for the image in FIG. 3( b) computedby using sample values scaled between [0, 255] for easy observation;

FIG. 3( d) presents a flowchart illustrating the steps of the methodperforming a level by level aggregation;

FIG. 3( e) presents a flowchart illustrating the steps of the methodperforming a cross-layer aggregation;

FIG. 4 presents a flow chart for illustrating steps of the method usedby an alternate embodiment using a SSIM-based computation of the measureof quality;

FIG. 5( a) displays LCC and SRCC between the DMOS and SSIM_(A)prediction values for various decomposition levels;

FIG. 5 (b) displays RMSE between the DMOS and SSIM_(DWT) predictionvalues for various β values;

FIG. 6( a) presents the block diagram for a prior art method based on aVIF index;

FIG. 6 (b) presents a flow chart for illustrating steps of the methodused by an alternate embodiment using a VIF-based computation of themeasure of quality;

FIG. 7( a) presents LCC and SRCC between the DMOS and VIF_(A) predictionvalues for various decomposition levels;

FIG. 7( b) displays RMSE between the DMOS and VIF_(DWT) predictionvalues for various 0 values;

FIG. 8 presents a flow chart for illustrating steps of the method usedby an alternate embodiment using a PSNR-based computation of the measureof quality;

FIG. 9( a) displays LCC and SRCC between the DMOS and PSNR_(A) qualityprediction values for various decomposition levels;

FIG. 9( b) displays RMSE between the DMOS and PSNR_(DWT) predictionvalues for various β values;

FIG. 9( c) displays the table showing SRCC for PSNR_(A) values fordifferent types of image distortion in the Live Image Database;

FIG. 10 presents a flow chart for illustrating steps of the method usedin an alternate embodiment using an AD-based computation of the measureof quality;

FIG. 11 displays LCC and SRCC between the DMOS and mean AD_(A)prediction values for various decomposition levels;

FIG. 12 presents LCC between different metrics and DMOS values of theLIVE database;

FIG. 13 shows RMSE between different metrics and DMOS values of the LIVEdatabase; and

FIG. 14 shows SRCC between metrics and DMOS values of the LIVE database.

DESCRIPTION OF THE EMBODIMENTS OF THE INVENTION

Terminology:

Contrast map: weighting function for automatically assigning weights todifferent regions of an image or sub-image based on their visualimportance.

Differential Mean Opinion Score (DMOS): A DMOS value indicates thesubjective quality loss of the distorted image compared to its referenceimage.

Discrete wavelet transform: transform applied to an image to separateits low frequency components from its high frequency components.

Edge map: characterizes an estimate for the edges of an image.

Gaussian sliding window: a set of coefficients with unit sum andGaussian probability distribution.

Linear Correlation Coefficient (LCC): Linear or Pearson Correlationcoefficient is the measure of dependence between two quantities. It isobtained by dividing the covariance of the two variables by the productof their standard deviations. In the proposed method LCC indicatesprediction accuracy.

Multiresolution decomposition: a process (or transform) that is appliedon digital images and generates several sub-images. One sub-imagerepresents the low frequency main content of original image and othersub-images show the details of the original image.

Spearman's Rank Correlation Coefficient (SRCC): SRCC is a measure ofstatistical dependence between two variables. It shows how well therelationship between the two variables can be described using amonotonic function. SRCC of +1 indicates that each of the variables is aperfect monotone function of the other.

Subbands: Various types of subbands are generated during the N levelmultiresoultion decomposition of an image. The description of thesesubbands and the terminology used for representing these for an image Xare presented. Please note that X and the index variable L used in thissection are mere examples and are replaced by other variable names inthe discussion presented in this application.

The subbands produced at the end of a N level multiresolutiondecomposition are called approximation subband and detail subbands thatare presented next.

Approximation subband: of an image X obtained after N level ofdecomposition is denoted by X_(A) _(N) (m,n). The approximation subbandcontains the main content of the image.

Detail subbands: of an image X are obtained after N level ofdecomposition and include a horizontal subband, a vertical subband and adiagonal subband. These are denoted respectively by: X_(H) _(N) (m,n),X_(V) _(N) (m,n), X_(D) _(N) (m,n).

The detail subbands contain the edges of the image. The horizontalsubband contains the horizontal edges, the vertical subband contains thevertical edges and the diagonal subbands contain the diagonal edges ofthe image.

Intermediate subbands: produced at each level L (L=1 to N−1) ofdecomposition for the image X include level L-wavelet packet (WP)subbands and level L-detail subbands. These are described next.

Level L-WP subbands: include level L-WP approximation subbands and levelL-WP detail subbands.

Level L-WP approximation subbands: include a level L-horizontal WPapproximation subband, a level L-vertical WP approximation subband, anda level L-diagonal WP approximation subband. These are denotedrespectively by:

X_(H) _(L) _(,A) _(N-L) (m,n), X_(V) _(L) _(,A) _(N-L) (m,n), X_(D) _(L)_(,A) _(N-L) (m,n).

Level L-WP detail subbands: include a level L-WP horizontal subband, alevel L-WP vertical subband and a level L-WP diagonal subband. These aredenoted respectively by:

X_(H) _(L) _(,H) _(N-L) (m,n), X_(V) _(L) _(,V) _(N-L) (m,n), X_(D) _(L)_(,D) _(N-L) (m,n).

Level L-detail subbands: include a level L-horizontal subband, a levelL-vertical, subband and a level L-diagonal subband. These are denotedrespectively by:

X_(H) _(L) (m,n), X_(V) L (m,n), X_(D) _(L) (m,n).

The embodiments of the present invention present a novel generalframework for determining a measure of quality for images moreaccurately, yet with less computational complexity, in comparison to theprior art in the discrete wavelet domain. Various measures of qualityeach based on a specific image quality metric (IQM) are determined bythe different embodiments of the invention. The proposed frameworkcovers both top-down and bottom-up approaches as described in thefollowing sections and can be applied to IQMs that include StructuralSimilarity (SSIM) index, absolute difference (AD), peak-signal-to-noiseratio (PSNR) and Visual Information Fidelity (VIF).

The systems of the embodiment of the invention are described next. Twoembodiments are presented in FIG. 1( a) and FIG. 1( k). Each of theseembodiments comprises units that are also described. The systems of theembodiments of the invention shown in FIG. 1( a) and FIG. 1( k) includea general purpose or specialized computer having a CPU and a computerreadable storage medium, e.g., memory, DVD, CD-ROM, floppy disk, flashmemory, magnetic tape or other storage medium, having computer readableinstructions stored thereon for execution by the CPU. Alternatively, thesystems of FIG. 1( a) and FIG. 1( k) can be implemented in firmware, orcombination of firmware and a specialized computer having a computerreadable storage medium.

The embodiment presented in FIG. 1( a) is presented first. A generalframework provided by the embodiment is discussed first followed by adiscussion on various other embodiments each of which deploys thisframework using a particular IQM.

The system 100 determining a measure of quality for an image comprises anumber of levels unit 102, a first decomposition unit 103, a seconddecomposition unit 104, an approximation quality measure unit 105, afirst details subband unit 106, a second details subband unit 107, aquality measure unit 108 and an edge quality measure unit 109. Thenumber of levels unit 102 determines the number of levels ofdecomposition, N, to be applied to the reference image X and thedistorted image Y. The number of levels of decomposition N is obtainedas a function of a minimum size of the approximation subband, S, whichproduces a substantially peak response for human visual system. Thevalue of N is then read by the first decomposition unit 103 and thesecond decomposition unit 104 that apply N level DWT to the image X andthe image Y respectively, thus performing N level decomposition of theimage X and the image Y. For each image, the N level decompositionproduces intermediate subbands at each level i (i=1 to N−1) forprocessing at level i+1 and detail subbands at level N. Theapproximation subbands generated after the multiresolution decompositionperformed by the first decomposition unit 103 and the seconddecomposition unit 104 are fed to the approximation quality measure unit105. The detail subbands generated after the multiresolutiondecomposition performed by the first decomposition unit 103 and thesecond decomposition unit 104 are used as inputs for the first detailsubbands unit 106 and the second details subband unit 107 that produceedge maps for the image X and the image Y respectively. The outputs ofthese units are connected to the edge quality measure unit 109, whichprocesses the edge maps, and produces an output used by the qualitymeasure unit 108 that determines the measure of quality.

The number of levels unit 102 in turn comprises computational means fordetermining the number of levels 110. The first decomposition unit 103and the second decomposition unit 104 comprise computational means forapplying N level DWT (111 and 112 respectively). The first detailsubbands unit 106 comprises a first aggregation unit 113 and the seconddetail subbands unit 107 comprises a second aggregation unit 114. Thequality measure unit 108 comprises a first pooling unit 115, a secondpooling unit 118, a contrast map unit 116 and a score combination unit117. The first pooling unit 115 and the second pooling unit 118 use acontrast map produced by the contrast map unit 116 for performingweighted pooling of the approximation quality map generated by theapproximation quality measure unit 105 and the edge map generated by theedge quality measure unit 109. The output of the first pooling unit 115and the second pooling unit 118 are used by the score combination unit117 to produce the measure of quality.

The structure of the number of levels unit 102 for another embodiment ofthe invention is presented in FIG. 1( b). It comprises a minimum sizeunit 120 determining, a ratio between viewing distance and a height of adevice on which the image Y is displayed and a computation unit 121 forusing this ratio and determining the value of N. In an alternateembodiment, the first decomposition unit 103 comprises (see FIG. 1( c))a first level 1 decomposition unit 124 and a first level j decompositionunit 125 and a first subband selection unit 126 determining the selectedintermediate subbands based on accuracy. The first subband selectionunit 126 is used for determining the intermediate subbands to be used bythe first level 1 decomposition unit 124 and the first level jdecomposition unit 125. The first subband selection unit 126 furthercomprises a first subband selection sub-unit 127 determining theselected intermediate subbands based on a number of the intermediatesubbands required for achieving the accuracy. As discussed in a latersection of this application, another selection process is used forselecting which of the subbands produced by the N level multiresolutiondecomposition are to be used during subband aggregation. The output ofthe first level 1 decomposition unit 124 that applies themultiresolution decomposition to the Image X producing intermediatesubbands at the level 1 for processing at level 2 is fed to the input ofthe first level j decomposition unit 125 that applies themultiresolution decomposition to only selected intermediate subbandsproduced by the multiresolution decomposition performed at the levelj−1, with j ranging from 2 to N. Similarly the second decomposition unit104 in turn comprises (see FIG. 1( d)) a second level 1 decompositionunit 130 and a second level j decomposition unit 131 and a secondsubband selection unit 132 for determining the selected intermediatesubbands based on accuracy. The second subband selection unit 132 isused for determining the intermediate subbands to be used by the secondlevel 1 decomposition unit 130 and the second level j decomposition unit131. The second subband selection unit 132 further comprises a secondsubband selection sub-unit 133 determining the selected intermediatesubbands based on a number of the intermediate subbands required forachieving the accuracy. The output of the second level 1 decompositionunit 130 that applies the multiresolution decomposition to the Image Yproducing intermediate subbands at the level 1 for processing at level 2is fed to the input of the second level j decomposition unit 131,applying the multiresolution decomposition to only selected intermediatesubbands produced by the multiresolution decomposition performed at thelevel j−1, with j ranging from 2 to N.

The components of the first aggregation unit 113 are presented in FIG.1( e). The first aggregation unit 113 comprises a first detail subbandsaggregation unit 135, a first level j aggregation unit 136 andcomputational means for choosing the selected intermediate subbands ateach level j and the selected detail subbands for the image X havingsubstantially the same resolution 134. The first level j aggregationunit 136 determines an edge map at the level j for the image X as afunction of the square of selected intermediate subbands produced atsaid each level j (j=1 to N−1). Please note that only some of theintermediate subbands produced at the level j, for example subbandshaving substantially the same resolution, are chosen for aggregationleading to the edge map at the level j. The output of the first level jaggregation unit 136 is processed by the first detail subbandsaggregation unit 135 that aggregates one or more of selected detailsubbands of the image X produced at level N with one or more of theselected intermediate subbands produced at each level j. Thecomputational means for choosing the selected intermediate subbands ateach level j and the selected detail subbands for the image X havingsubstantially same resolution 134 are used by the first detail subbandsaggregation unit 135 and the first level j aggregation unit 136 forselecting the intermediate subbands at each level j and the detailsubbands for the image X having substantially the same resolution foraggregating. Please note that j is merely an index variable and canassume any integer value between 1 and N−1. The first detail subbandsaggregation unit 135 in turn comprises a first level N aggregation unit137 and a first edge map unit 138. The first level N aggregation unit137 determines an edge map at the level N for the image X as a functionof the square of the selected detailed subbands of the image X producedat the level N whereas the first edge map unit 138 determines the edgemap of the image X as a weighted sum of the edge maps at the level j andthe edge map at the level N that is received from the first level Naggregation unit 137.

The components of the second aggregation unit 114 are presented in FIG.1( f). The second aggregation unit 114 comprises a second detailsubbands aggregation unit 140, a second level j aggregation unit 141 andcomputational means for choosing the selected intermediate subbands ateach level j and the selected detail subbands for the image Y havingsubstantially same resolution 139. The second level j aggregation unit141 determines an edge map at each level j (j=1 to N−1) for the image Yas a function of the square of the selected intermediate subbandsproduced at said each level j. The output of the second level jaggregation unit 141 is processed by the second detail subbandsaggregation unit 140 that aggregates one or more of selected detailsubbands of the image Y produced at level N with one or more of theselected intermediate subbands produced at each level j. Thecomputational means for choosing the selected intermediate subbands ateach level j and the selected detail subbands for the image Y havingsubstantially same resolution 139 are used by the second detail subbandsaggregation unit 140 and the second level j aggregation unit 141 forselecting the intermediate subbands at each level j and the detailsubbands for the image Y having substantially the same resolution foraggregating. Once again, please note that j is merely an index variableand can assume any integer value between 1 and N−1. The second detailsubbands aggregation unit 140 in turn comprises a second level Naggregation unit 142 and a second edge map unit 143. The second level Naggregation unit 142 that determines an edge map at the level N for theimage Y as a function of the square of the selected detailed subbands ofthe image Y produced at the level N whereas the second edge map unit 143determines the edge map of the image Y as a weighted sum of the edgemaps at the level j and the edge map at the level N that is receivedfrom the second level N aggregation unit 142.

The general framework provided by the system 100 is used in alternateembodiments each of which uses a specific IQM in determining the measureof quality. The system of the embodiment based on the SSIM IQM ispresented in FIG. 1( g). The approximation quality measure unit 105 inthis embodiment comprises an approximation SSIM unit 145, the qualitymeasure unit 108 comprises a SSIM_(DWT) score unit 146 and the edgequality measure unit 109 comprises an edge SSIM map unit 147. Theapproximate SSIM map unit 145 applies a SSIM IQM to the approximationsubband of the image X and the approximation subband of the image Y toproduce an approximation SSIM map whereas the edge SSIM map unit 147applies the SSIM IQM between the edge map of the image X and the edgemap of the image Y to produce an edge SSIM map. The outputs of theapproximation SSIM map unit 145 and the edge SSIM map unit 147 areprocessed by the SSIM_(DWT) score unit 146 to determine a SSIM_(DWT)score as the measure of quality.

The system of the embodiment based on the AD IQM is presented in FIG. 1(h). The approximation quality measure unit 105 in this embodimentcomprises an approximation AD map unit 150, the quality measure unit 108comprises an AD_(DWT) score unit 151 and the edge quality measure unit109 comprises an edge AD map unit 152. The approximate AD map unit 150applies an AD IQM to the approximation subband of the image X and theapproximation subband of the image Y to produce an approximation AD mapwhereas the edge AD map unit 152 applies the AD IQM between the edge mapof the image X and the edge map of the image Y to produce an edge ADmap. The outputs of the approximation AD map unit 150 and the edge ADmap unit 152 are processed by the AD_(DWT) score unit 151 to determinean AD_(DWT) score as the measure of quality.

The system of the embodiment based on the PSNR IQM is presented in FIG.1( i). The approximation quality measure unit 105 in this embodimentcomprises a PSNR approximation quality score unit 155, the qualitymeasure unit 108 comprises a PSNR_(DWT) score unit 156 and the edgequality measure unit 109 comprises a PSNR edge quality scoredetermination unit 157. The PSNR approximation quality score unit 155applies the PSNR IQM to the approximation subband of the image X and theapproximation subband of the image Y to produce a PSNR approximationquality score whereas the PSNR edge quality score determination unit 157applies the PSNR IQM between the edge map of the image X and the edgemap of the image Y to produce a PSNR edge quality score. The outputs ofthe PSNR approximation quality score unit 155 and the PSNR edge qualityscore determination unit 157 are processed by the PSNR_(DWT) score unit156 to determine a PSNR_(DWT) score as the measure of quality.

The system of the embodiment based on the VIF IQM is presented in FIG.1( j). The approximation quality measure unit 105 in this embodimentcomprises a VIF approximation quality score unit 160, the qualitymeasure unit 108 comprises a VIF_(DwT) score unit 161 and the edgequality measure unit 109 comprises a VIF edge quality scoredetermination unit 162. The VIF approximation quality score unit 160applies the VIF IQM to the approximation subband of the image X and theapproximation subband of the image Y to produce a VIF approximationquality score whereas the VIF edge quality score determination unit 162applies the VIF IQM between the edge map of the image X and the edge mapof the image Y to produce a VIF edge quality score. The outputs of theVIF approximation quality score unit 160 and the VIF edge quality scoredetermination unit 162 are processed by the VIF_(DWT) score unit 161 todetermine a VIF_(DWT) score as the measure of quality.

All the components of the system 100 that include units and sub-units100, 102, 103, 104, 105, 106, 107, 108, 109, 113, 114, 115, 116, 117,118, 120, 121, 124, 125, 126, 127, 130, 131, 132, 133, 135, 136, 137,138, 140, 141, 142, 143, 145, 146, 147, 150, 151, 152, 155, 156, 157,160, 161 and 162 include a firmware or, alternatively, computer readableinstructions stored in a computer readable storage medium for executionby a processor. All the computational means including 110, 111, 112, 134and 139 comprise computer readable code performing methods, procedures,functions or subroutines which are stored in a computer readable storagemedium to be executed by a CPU.

An alternate embodiment for the general framework presented in FIG. 1(k) is discussed next. System 100 a determining a measure of quality forimages comprises computational means for determining N 110 a, a firstdecomposition unit 103 a, a second decomposition unit 104 a, anapproximation quality measure unit 105 a, a first aggregation unit 113a, a second aggregation unit 114 a, a quality measure unit 108 a and anedge quality measure unit 109 a. The units 103 a, 104 a, 105 a, 108 a,109 a, 115 a, 116 a, 117 a and 118 a are the same as the units 103, 104,105, 108, 109, 115, 116, 117 and 118 of system 100 and are not discussedany further. The computational means 110 a, 111 a, 112 a, 134 a and 139a are the same as the computational means 110, 111, 112, 134 and 139 ofsystem 100 presented earlier.

As shown in FIG. 1( l), the first aggregation unit 113 a processingimage X comprises a first selection unit 170 the output of which isconnected to the input of the first selective aggregation unit 171. Thefirst selection unit 170 selects the intermediate subbands at each leveli, with i ranging from 1 to N−1, and the detail subbands for the image Xbased on an accuracy to be achieved in determining the measure ofquality. The first selective aggregation unit 171 aggregates only theselected intermediate subbands and the selected detail subbands for theimage X. The first aggregation unit 113 a further comprisescomputational means for choosing the selected intermediate subbands ateach level i and the selected detail subbands for the image X havingsubstantially same resolution 134 a. 134 a is similar to 134 discussedearlier and is used for selecting the intermediate subbands and thedetail subbands that have substantially the same resolution foraggregating. The first selection unit 170 further comprises a firstselection sub-unit 172 that selects the intermediate subbands at eachlevel i and the detail subbands for the image X based on a number of theintermediate and the detailed subbands required for achieving theaccuracy. In another alternate embodiment, the first selection unitcomprises a first wavelet packet (WP) detail selection sub-unit 176 thatselects one or more of the level i-WP approximation subbands and one ormore of the detail subbands for the image X. The first WP detailsubbands selection sub-unit 176 in turn comprises a first detailselection module 179 for including one or more of the level i-detailsubbands for the image X in the selecting.

The first level i-intermediate subbands aggregation unit 173 comprises afirst level i computation unit 177 for squaring each selectedintermediate subband for the image X for each level i (i=1 to N−1);multiplying a result of the squaring performed with a predeterminedweight for said each selected intermediate subband for the image X;summing a product resulting from the multiplying performed in the step;and applying a square root function to a result of the summing performedproducing the edge map at the level i for the image X. The first detailsubbands aggregation unit 174 comprises a first detail subbandscomputation unit 180 for squaring each said selected detail subband forthe image X; multiplying a result of the squaring performed in the stepwith a predetermined weight for said each selected detail subband forthe image X; summing a product resulting from the multiplying performedin the step; and applying a square root function to the result of thesumming performed in the step producing the edge map at the level N forthe image X. The first edge map determination unit 175 comprises a firstcomputation unit 178 for: multiplying the edge map at each level i forthe image X with a predetermined weight for the level i; multiplying theedge map at the level N for the image X with a predetermined weight atthe level N; and summing products resulting from the multiplying.

As shown in FIG. 1( m) the second aggregation unit 114 a processingimage Y comprises a second selection unit 182 the output of which isconnected to the input of the second selective aggregation unit 183. Thesecond selection unit 182 selects the intermediate subbands at eachlevel i, with i ranging from 1 to N−1, and the detail subbands for theimage Y based on an accuracy to be achieved in determining the measureof quality. The second selective aggregation unit 183 aggregates onlythe selected intermediate subbands and the selected detail subbands forthe image Y. The second aggregation unit 114 a further comprisescomputational means for choosing the selected intermediate subbands ateach level i and the selected detail subbands for the image Y havingsubstantially same resolution 139 a. 139 a is similar to 139 discussedearlier and is used for selecting the intermediate subbands and thedetail subbands that have substantially the same resolution foraggregating. The second selection unit 182 further comprises a secondselection sub-unit 184 that selects the intermediate subbands at eachlevel i and the detail subbands for the image Y based on a number of theintermediate and the detailed subbands required for achieving theaccuracy. In yet another alternate embodiment, the second selection unit182 comprises a second WP detail selection sub-unit 188 that selects oneor more of the level i-WP approximation subbands and one or more of thedetail subbands for the image Y. The second WP detail subbands selectionsub-unit 188 in turn comprises a second detail selection module 191 forincluding one or more of the level i-detail subbands for the image Y inthe selecting.

The second level i-intermediate subbands aggregation unit 185 comprisesa second level i computation unit 189 for squaring each selectedintermediate subband for the image Y for said each level i; multiplyinga result of the squaring performed in the step with a predeterminedweight for said each selected intermediate subband for the image Y;summing a product resulting from the multiplying performed; and applyinga square root function to a result of the summing performed producingthe edge map at the level i for the image Y. The second detail subbandsaggregation unit 186 comprises a second detail subbands computation unit192 for squaring each said selected detail subband for the image Y;multiplying a result of the squaring performed with a predeterminedweight for said each selected detail subband for the image Y; summing aproduct resulting from the multiplying performed; and applying a squareroot function to a result of the summing performed producing the edgemap at the level N for the image Y. The second edge map determinationunit 187 comprises a second computation unit 190 for: multiplying theedge map at each level i for the image Y with a predetermined weight forthe level i; multiplying the edge map at the level N for the image Ywith a predetermined weight at the level N; and summing productsresulting from the multiplying.

The structure of the first selective aggregation unit 171 is presentedin FIG. 1( n). The first selective aggregation unit comprises a firsthorizontal edge map unit 194, a first vertical edge map unit 195, and afirst diagonal edge map unit 196 and a first combination unit 197. Theintermediate subbands produced at the level i of decomposition includewavelet packet (WP) approximation and wavelet packet detail subbands.Each of these units, 194, 195 and 196 processes a particular type of WPapproximation subband and WP detail subband. These subbands arediscussed in more detail in a later part of this document. The firsthorizontal edge map unit 194 aggregates the level i-horizontal WPapproximation subbands for the image X at each level i (i=1 to N−1) andthe horizontal subband for the image X producing a horizontal edge mapfor the image X. The first vertical edge map unit 195 aggregates thelevel i-vertical WP approximation subbands for the image X at each leveli and the vertical subband for the image X producing a vertical edge mapfor the image X. The first diagonal edge map unit 196 aggregates thelevel i-diagonal WP approximation subbands for the image X at each leveli and the diagonal subband for the image X producing a diagonal edge mapfor the image X. The horizontal, vertical and the diagonal edge maps ofthe image X are aggregated by the first combination unit 197.

The level i-WP detail subbands for the image X further comprise for eachrespective image, a level i-WP horizontal subband, a level i-WP verticalsubband and a level i-WP diagonal subband. The first horizontal edge mapunit 194 comprises a first horizontal sub-unit 198 that includes thelevel i-WP horizontal subband for the image X in the aggregating. Thefirst vertical edge map unit 195 comprises a first vertical sub-unit 199that includes the level i-WP vertical subband for the image X in theaggregating. The first diagonal edge map unit 196 comprises a firstdiagonal sub-unit 200 that includes the level i-WP diagonal subband forthe image X in the aggregating. The first combination unit 197 in turncomprises a first combination sub-unit 201 applying a square rootfunction to the horizontal edge map, the vertical edge map and thediagonal edge map for the image X and summing results of applying thesquare root function.

The structure of the second selective aggregation unit 183 is presentedin FIG. 1( o). The second selective aggregation unit comprises a secondhorizontal edge map unit 203, a second vertical edge map unit 204, asecond diagonal edge map unit 205 and a second combination unit 206. Thesecond horizontal edge map unit 203 aggregates the level i-horizontal WPapproximation subbands for the image Y at each level i (i=1 to N−1) andthe horizontal subband for the image Y producing a horizontal edge mapfor the image Y. The second vertical edge map unit 204 aggregates thelevel i-vertical WP approximation subbands for the image Y at each leveli and the vertical subband for the image Y producing a vertical edge mapfor the image Y. The second diagonal edge map unit 205 aggregates thelevel i-diagonal WP approximation subbands for the image Y at each leveli and the diagonal subband for the image Y producing a diagonal edge mapfor the image Y. The horizontal, vertical and the diagonal edge maps ofthe image Y are aggregated by the second combination unit 206.

As in the case of image X, the level i-WP detail subbands for the imageY further comprise for each respective image, a level i-WP horizontalsubband, a level i-WP vertical subband and a level i-WP diagonalsubband. The second horizontal edge map unit 203 comprises a secondhorizontal sub-unit 207 that includes the level i-WP horizontal subbandfor the image Y in the aggregating. The second vertical edge map unit204 comprises a second vertical sub-unit 208 that includes the leveli-WP vertical subband for the image Y in the aggregating. The seconddiagonal edge map, unit 205 comprises a second diagonal sub-unit 209that includes the level i-WP diagonal subband for the image Y in theaggregating. The second combination unit 206 in turn comprises a secondcombination sub-unit 210 applying a square root function to thehorizontal edge map, the vertical edge map and the diagonal edge map forthe image Y and summing results of applying the square root function.

In an alternate embodiment a different set of mathematical operations isperformed during the aggregation performed by the first combination unit197 and the second combination unit. This embodiment first combinationunit comprises a third combination sub-unit 211 (see FIG. 1( p)) and thesecond combination unit 206 comprises a fourth combination sub-unit 212(see FIG. 1( q)). The third combination sub-unit 211 is used for summingthe horizontal edge map, the vertical edge map and the diagonal edge mapfor the image X and applying a square root function to result of thesumming performed. The fourth combination sub-unit 212 is used forsumming the horizontal edge map, the vertical edge map and the diagonaledge map for the image Y and applying a square root function to resultof the summing performed.

The first decomposition unit 103 a shown in FIG. 1( r) comprises a firstlevel 1 decomposition unit 214 the output of which is used by a firstlevel i decomposition unit 215. The first level 1 decomposition unit 214applies the multiresolution decomposition to the image X producingintermediate subbands at the level 1 for processing at level 2 whereasthe first level i decomposition unit 215 applies the multiresolutiondecomposition to the selected intermediate subbands produced by themultiresolution decomposition performed at the level i−1, with i rangingfrom 2 to N.

The second decomposition unit 104 a shown in FIG. 1( s) comprises asecond level 1 decomposition unit 216 the output of which is used by asecond level i decomposition unit 217. The second level 1 decompositionunit 216 applies the multiresolution decomposition to the image Yproducing intermediate subbands at the level 1 for processing at level 2whereas the second level i decomposition unit 217 applies themultiresolution decomposition to the selected intermediate subbandsproduced by the multiresolution decomposition performed at the leveli−1, with i ranging from 2 to N.

As in the case of the general framework for the system 100, the generalframework of the system 100 a can be used in conjunction with variousIQMs each of which leads to a specific embodiment.

All the components of the system 100 a that include units and sub-unitsand modules labeled with reference numerals 103 a, 104 a, 105 a, 108 a,109 a, 113 a, 114 a, 115 a, 116 a, 117 a, 118 a, 170,171, 172, 173, 174,175, 176, 177, 178, 179, 180, 182, 183, 184, 185, 186, 187, 188, 189,190, 191, 192, 194, 195, 196, 197, 198, 199, 200, 201, 203, 204, 205,206, 207, 208, 209, 210, 211, 212, 214, 215, 216 and 217 include afirmware or, alternatively, computer readable instructions stored in acomputer readable storage medium for execution by a processor. All thecomputational means labeled with reference numerals 110 a, 111 a,112 a,134 a and 139 a comprise computer readable storage medium havingcomputer readable code for execution by a CPU, performing methods,procedures, functions or subroutines as described in this application.

A high level description of the technique used by the framework todetermine the measure of quality is captured in diagram 220 displayed inFIG. 2( a). The procedure starts with performing a N level DWT on thereference image (box 222) and the distorted image (box 224). Themultiresolution decomposition achieved through the application of theDWT produces approximation subbands and detail subbands for both theimages. The approximation subbands of both the images are processed tocompute an approximation quality measure (box 232). Please note thatdepending on the IQM used this quality measure can be a quality map or aquality score. The detail subbands of both the original and thedistorted image are aggregated (box 226 and box 230). These aggregatesare then processed to compute an edge quality measure, map or score (box234). The map or index determined depends on the IQM used in thegeneration of the measure of quality and is discussed in detail in alater section. For certain IQMs (described in a later section) acontrast map needs to be generated (box 228). Generating a contrast map,includes assigning corresponding values to the pixels of theapproximation subband and the edge map of the image according to theirrespective importance to human visual system. The contrast map is usedin pooling of the approximation quality map (box 236) and the pooling ofthe edge quality map (box 238). The results of the two poolingoperations are then combined to produce the measure of quality for thedistorted image (box 240). With IQMs, for which the contrast map is notused, the approximation quality measure, and the edge quality measure,from box 232 and box 234 respectively, are combined by box 240 toproduce the measure of quality.

Two selection processes are used in the invention. The first is used atlevel i of decomposition (i=1 . . . N−1) for selecting the intermediatesubbands produced by the multiresolution decomposition performed at thelevel i−1 of the image X or Y on which the multiresolution decompositionwill be applied at the level i. The second is for selecting theintermediate subbands at each level i, and the detail subbands for theimage X or Y that are to be aggregated. The selections are performed ina way that strikes an effective tradeoff between accuracy andcomputational overhead. The selection processes can be based on a numberof the subbands required for achieving the accuracy. The method of theembodiment based on the technique deployed in the framework is explainedwith the help of FIG. 2( b). Upon start (box 252), the procedure 250computes N, the number of levels for the DWT (box 254). N can becomputed by using one or more of a resolution (or size) of the image Y,a viewing distance for the image Y, and an image quality metric (IQM)applied to the image Y. The procedure 250 determines this number oflevels of decomposition N as a function of a minimum size of theapproximation subband, S, which produces a substantially peak responsefor human visual system. Thus, the value of N is determined in such away that it leads to a resolution for the approximation subband at whichthe corresponding maximum frequency is close to the peak sensitivity ofthe human visual system (i.e. the maximum value of the spatial frequencyresponse of the human visual system).

As discussed by Y. Wang, J. Ostermann and Y.-Q. Zhang in “VideoProcessing Communications”, Prentice Hall, 2001, the spatial frequencyresponse is typically a curve of the human contrast sensitivity as afunction of the spatial frequency represented in cycles per degree(CPD). This peak is between 2 and 4. The values of N used are tailoredto the specific IQM used as discussed in later sections of thisapplication. For certain IQMs N is a fixed value, but for error-basedIQMs, such as PSNR or absolute difference (AD), the required number ofdecomposition levels, N, is formulated as follows. As described by M. R.Bolin, and G. W. Meyer, “A visual difference metric for realistic imagesynthesis”, in Proc. SPIE Human Vision, Electron. Imag., vol. 3644, SanJose, 1999, pp. 106-120, when an image is viewed at the distance d of adisplay of height h, we have:

$\begin{matrix}{f_{\theta} = {\frac{\pi}{180}\frac{d}{h}f_{s}\mspace{14mu}\left( {{cycles}\text{/}{degree}} \right)}} & \left( {1a} \right)\end{matrix}$

Where f_(θ) is the angular frequency which has a unit of cycle/degree(cpd), and f_(s) denotes the spatial frequency. For an image of heightH, the Nyquist theorem results in eq. (1b).

$\begin{matrix}{\left( f_{s} \right)_{\max} = {\frac{H}{2}\mspace{14mu}\left( {{cycles}\text{/}{picture}\text{-}{height}} \right)}} & \left( {1b} \right)\end{matrix}$

It is known that the HVS has a peak response for frequencies at about2-4 cpd. So f_(θ) is fixed at 3. If the image is assessed at viewingdistance of d=kh, using eq. (1a) and eq. (1b) yields eq. (1c).

$\begin{matrix}{{H \geq \frac{360f_{\theta}}{\pi\left( {d/h} \right)}} = {\frac{360 \times 3}{3.14 \times k} \approx \frac{344}{k}}} & \left( {1c} \right)\end{matrix}$

So, the effective size of an image for human eye assessment should bearound (344/k). This implies that the minimum size of approximationsubband after N-level decomposition should be approximately equal to(344/k). Thus, for an image of size H×W, N is approximately calculatedas follows (considering that N must be non negative):

$\begin{matrix}{\left. {\frac{\min\left( {H,W} \right)}{2^{N}} \approx \frac{344}{k}}\Rightarrow N \right. = {{round}\left( {\log_{2}\left( \frac{\min\left( {H,W} \right)}{\left( {344/k} \right)} \right)} \right)}} & \left( {1d} \right) \\{\left. {N \geq 0}\Rightarrow N \right. = {\max\left( {0,{{round}\left( {\log_{2}\left( \frac{\min\left( {H,W} \right)}{\left( \frac{344}{k} \right)} \right)} \right)}} \right)}} & \left( {1e} \right)\end{matrix}$

After determining N, a N level DWT using the Haar filter, for example,is performed on both the reference (image X) and distorted (image Y)images (box 256). With N level decomposition, the approximation subbandsX_(A) _(N) , and Y_(A) _(N) as well as a number of detail subbands, areobtained. For simplicity we refer to X_(A) _(N) and Y_(A) _(N) as X_(A)and Y_(A) respectively. Note that the method of the embodiments of theinvention can readily use various other various wavelet filters. TheHaar filter was chosen for its simplicity and good performance. Itssimplicity imposes negligible computational burden on the algorithm.Based on simulations performed by the applicants, the Haar wavelet alsoprovides more accurate measures of quality in comparison to otherwavelet bases. The reason is that symmetric Haar filters have ageneralized linear phase, so the perceptual image structures can bepreserved. Also, Haar filters avoid over-filtering the image due totheir short filter length.

In the next step, the procedure 250 computes the approximation qualitymeasure, IQM_(A), by applying an IQM between the approximation subbandsof image X, X_(A) and the approximation subband of image Y, Y_(A), (box258). The edge maps for image X and image Y are generated next (box260). Please note that the edge maps only reflect the edge structures ofimages. An estimate of the image edges is formed for each image using anaggregate of the detail subbands. Various methods of performingaggregation of the detail subbands are used in multiple embodiments ofthe invention. One such method that performs a level by levelaggregation is presented next. Alternate methods of subband aggregationsuch as cross-layer aggregation the edge maps of image X and image Y aredeployed in alternate embodiments of this invention and are discussed ina later section of this invention.

With a level by level aggregation, aggregation is performed at eachlevel i (i=1 to N). When a N level DWT is applied to the images, theedge map (edge estimate of the image) of image X is defined as:

${X_{E}\left( {m,n} \right)} = {\sum\limits_{L = 1}^{N}{X_{E,L}\left( {m,n} \right)}}$where X_(E) is the edge map of X, and X_(E,L) is the edge map at thedecomposition level L, as defined in eq. (2). In eq. (2), X_(H) _(L) ,X_(V) _(L) , and X_(D) _(L) denote the horizontal, vertical, anddiagonal detail subbands obtained at the decomposition level L for imageX. An approximation subband contains main content of an image while thedetail subbands contain edges of the image X. X_(H) _(L) _(,A) _(N-L) ,X_(V) _(L) _(,A) _(N-L) , and X_(D) _(L) _(,A) _(N-L) are wavelet packetapproximation subbands, obtained by applying an (N-L)-level DWT on X_(H)_(L) , X_(V) _(L) , and X_(D) _(L) respectively. These are included inthe intermediate subbands and are referred to as level L-WPapproximation subbands. The parameters μ, λ, and ψ are weightsassociated with the various subbands used in eq. (2). As the HVS is moresensitive to the horizontal and vertical subbands and less sensitive tothe diagonal one, greater weight is given to the horizontal and verticalsubbands and a smaller weight to the diagonal subband. μ=λ=4.5ψ is usedin the embodiment of this invention. This results in μ=λ=0.45 and ψ=0.10that satisfy eq. (3). The embodiments of the invention are flexible toaccommodate various other values of these weights.

$\begin{matrix}{{X_{E,L}\left( {m,n} \right)} = \left\{ \begin{matrix}{\sqrt{{\mu \cdot \left( {X_{H_{L}}\left( {m,n} \right)} \right)^{2}} + {\lambda\left( {X_{V_{L}}\left( {m,n} \right)} \right)}^{2} + {\psi\left( {X_{D_{L}}\left( {m,n} \right)} \right)}^{2}},{{{if}\mspace{14mu} L} = N}} \\{\sqrt{{\mu \cdot \left( {X_{H_{L},A_{N - L}}\left( {m,n} \right)} \right)^{2}} + {\lambda\left( {X_{V_{L},A_{N - L}}\left( {m,n} \right)} \right)}^{2} + {\psi\left( {X_{D_{L},A_{N - L}}\left( {m,n} \right)} \right)}^{2}},} \\{{{if}\mspace{14mu} L} < N}\end{matrix} \right.} & (2) \\{\mspace{20mu}{{\mu + \lambda + \psi} = 1}} & (3)\end{matrix}$

The edge map of Y is defined in a similar way as X. As an example, table300 in FIG. 3( a 1) displays the subbands of image X for N=2. Onlyselected subbands are used in generating the edge map. The subbandsinvolved in computing the edge map are enclosed in double boxes in thisfigure. As discussed earlier, the edge map is intended to be an estimateof image edges. Thus, rather than considering all of them, the mostinformative subbands are used in forming the edge maps and computationaloverhead is reduced. According to simulations performed by theapplicants, using all image subbands in computing the edge maps does nothave a significant impact on increasing the accuracy of the measure ofquality. As another example, table 302 in FIG. 3( a 2) displays thesubbands of image X for N=3. Only selected subbands are used ingenerating the edge map. The subbands involved in computing the edge mapare enclosed in solid double boxes in this figure. When extended to Nequal to or greater than 2 the technique used for selecting the subbandsshown in FIG. 3( a 1) and FIG. 3( a 2) will lead to the selection ofonly 3N subbands. If all the generated subbands are used the generationof the edge map one would have to use 4^(N)−1 subbands. When N isgreater than or equal to 2, 4^(N)−1 is much greater than 3N. Thus, thetechnique used by the embodiments of the invention lead to a greatreduction in computational complexity. After generating the edge mapsfor the image X and the image Y, the IQM is applied between the edgemaps X_(E) and Y_(E) (box 262). The resulting quality measure is calledthe edge quality measure, IQM_(E). The term map or score to be useddepending on the specific IQM used, as discussed in the followingsections of this application. The procedure 250 then checks whether ornot a contrast map needs to be generated (box 264). If so, the procedureexits ‘YES’ from box 264, and generates the contrast map (box 266). Sucha contrast map is required in case of a number of IQMs. Certain IQMsincluding Absolute Difference (AD) and Structural Similarity Matrix(SSIM) generate intermediary quality maps that should be pooled to reachrespective quality scores. In this step, the procedure 250 forms acontrast map for pooling the approximation and the edge quality maps. Itis well known that the HVS is more sensitive to areas near the edges andis discussed by Z. Wang, and A. C. Bovik, in “Modern Image QualityAssessment”, USA: Morgan & Claypool, 2006. Therefore, the pixels in thequality map near the edges are given more importance. In addition, asdiscussed by Z. Wang, and X. Shang, in “Spatial Pooling Strategies forPerceptual Image Quality Assessment”, Proc. IEEE Int. Conf. ImageProcess., Atlanta, October 2006, pp. 2945-2948, high-energy (orhigh-variance) image regions are likely to contain more information toattract the HVS. Thus, the pixels of the quality map within high-energyregions must also receive higher weights (more importance). Based onthese facts, the edge map is combined with the computed variance to forma function called the contrast map. The contrast map is computed withina local Gaussian square window, which moves (pixel-by-pixel) over theentire edge maps X_(E) and Y_(E). As discussed by Z. Wang, A. Bovik, H.Sheikh, and E. Simoncelli, in “Image quality assessment: From errorvisibility to structural similarity”, IEEE Transactions on ImageProcessing, vol. 13, no. 4, pp. 600-612, April 2004, a Gaussian slidingwindow W={w_(k)|k=1, 2, K, K} with a standard deviation of 1.5 samples,normalized to unit sum is used. The number of coefficients K is set to16, that is, a window of size 4×4 is used. This window size is not toolarge and can provide accurate local statistics. The contrast map isdefined as follows:

$\begin{matrix}{{{Contrast}\left( {x_{E},x_{A_{N}}} \right)} = \left( {\mu_{x_{E}}^{2}\sigma_{x_{A_{N}}}^{2}} \right)^{0.15}} & (4) \\{\sigma_{x_{A_{N}}}^{2} = {\sum\limits_{k = 1}^{K}{w_{k}\left( {x_{A_{N},k} - \mu_{x_{A_{N}}}} \right)}^{2}}} & (5) \\{{\mu_{x_{E}} = {\sum\limits_{k = 1}^{K}{w_{k}x_{E,k}}}},\mspace{14mu}{\mu_{x_{A_{N}}} = {\sum\limits_{k = 1}^{K}{w_{k}x_{A_{N},k}}}}} & (6)\end{matrix}$where x_(E) and x_(A) _(N) denote image patches of X_(E) and X_(A) _(N)within the sliding window. Please note that the contrast map exploitsthe original image statistics to form the weighted function for qualitymap pooling. Picture 330 displayed in FIG. 3( c) demonstrates a resizedcontrast map, obtained by eq. (4), for a typical image shown in picture320 presented in FIG. 3( b). As can be seen from FIG. 3( b) and FIG. 3(c), the contrast map clearly shows the edges and important imagestructures to the HVS. Brighter (higher) sample values in the contrastmap indicate image structures which are more important to the HVS andplay an important role in judging image quality. In the next step, theprocedure uses the contrast map defined in (4) to perform weightedpooling of the approximation quality map IQM_(A), and the edge qualitymap IQM_(E) (box 268):

$\begin{matrix}{S_{A} = \frac{\sum\limits_{j = 1}^{M}{{{Contrast}\left( {x_{E,j},x_{A_{N},j}} \right)} \cdot {{IQM}_{A}\left( {x_{A_{N},j},y_{A_{N},j}} \right)}}}{\sum\limits_{j = 1}^{M}{{Contrast}\left( {x_{E,j},x_{A_{N},j}} \right)}}} & (7) \\{S_{E} = \frac{\sum\limits_{j = 1}^{M}{{{Contrast}\left( {x_{E,j},x_{A_{N},j}} \right)} \cdot {{IQM}_{E}\left( {x_{E,j},y_{E,j}} \right)}}}{\sum\limits_{j = 1}^{M}{{Contrast}\left( {x_{E,j},x_{A_{N},j}} \right)}}} & (8)\end{matrix}$where x_(E,j) and X_(A) _(N) _(,J) in the contrast map denote imagepatches in the j-th local window; x_(A) _(N) _(,j), y_(A) _(N) _(,j),x_(E,j), and y _(E,j) terms in the quality map are image patches (orpixels) in the j-th local window position; M is the number of pixels inthe quality maps; S_(A) and S_(E) represent the approximation and edgequality scores respectively achieved after the weighted poolingoperation.

The measure of quality for the image Y is determined next (box 270). Inthis step, the approximation and edge quality scores are combinedlinearly, as defined in eq. (9), to obtain the overall quality scoreIQM_(DWT) between images X and Y.

$\begin{matrix}{{{{IQM}_{DWT}\left( {X,Y} \right)} = {{\beta \cdot S_{A}} + {\left( {1 - \beta} \right) \cdot S_{E}}}}{0 < \beta \leq 1}} & (9)\end{matrix}$where IQM_(DWT) gives the measure of quality for the image Y withrespect to image X, and is a constant. As the approximation subbandcontains main image contents, should be close to 1 to give theapproximation quality score (S_(A)) a higher importance. Aftergenerating the measure of quality the procedure 250 exits (box 272). Ifa particular embodiment of the invention uses an IQM that does notrequire a contrast map, the procedure 250 exits ‘No’ from box 264, skipsboxes 266 and 268, combines the approximation quality measure and theedge quality measure to determine the measure of quality (box 270) andexits (box 272).

As mentioned earlier the alternate embodiments of the invention useother methods of performing aggregation and are discussed next.

Alternate Embodiments Deploying Alternate Methods of Generating EdgeMaps

The methods used in performing aggregation of the detail subbands of thereference image X and the distorted image Y that are used in thegeneration of the edge maps for the image X and the image Y areexplained with the help, of FIG. 3( d) and FIG. 3( e). Before explainingthe steps of the method performing the aggregation a discussion of theunderlying concepts is presented.

An estimate of the image edges is formed for each image X and Y using anaggregate of the detail subbands. If the N-level DWT is applied to theimages, the edge map of image X is defined as in eq. (6).

$\begin{matrix}{{X_{E}\left( {m,n} \right)} = {\sum\limits_{L = 1}^{N}{\omega_{L}{X_{E,L}\left( {m,n} \right)}}}} & \left( {6a} \right)\end{matrix}$

With normally ω_(L)<=ω_(L+1) for all L

where X_(E) is the edge map of X, and X_(E,L) is the edge map at level Lfor image X, computed as defined in eq. (2). The weights ω_(L) are usedto associate various degrees of importance to the multiresolutiondecomposition performed at the different levels 1 to N. A set ofpredetermined weights are used for any level i (i=1 to N). Note that thevalue of L used in the equation is equal to i indicating the level.

There are other ways of aggregating the intermediate subbands producedat the levels 1 to N−1 and the detail subbands produced at the level Nof the multiresolution decomposition. The aggregation may be performedlevel by level, or alternatively, a cross-level aggregation may be alsoperformed. However, only subbands of the substantially same resolutionare to be combined during the aggregation process. The level by levelaggregation that was described earlier is generalized and presented forcontrasting with the cross-layer aggregation. With the layer by layermethod, aggregation of selected intermediate subbands is performed ateach level i (i=1 to N−1) producing an edge map at level i whereasaggregation of selected detail subbands is performed at level Nproducing an edge map at the level N. Thus, eq. (2) can be generalizedas:

$\begin{matrix}{{X_{E,L}\left( {m,n} \right)} = \left\{ \begin{matrix}{\sqrt{{\mu_{L} \cdot \left( {X_{H_{L}}\left( {m,n} \right)} \right)^{2}} + {\lambda_{L}\left( {X_{V_{L}}\left( {m,n} \right)} \right)}^{2} + {\psi_{L}\left( {X_{D_{L}}\left( {m,n} \right)} \right)}^{2}},} \\{{{if}\mspace{14mu} L} = N} \\{\sqrt{\begin{matrix}{{\mu_{L} \cdot \left( {X_{H_{L},A_{N - L}}\left( {m,n} \right)} \right)^{2}} +} \\{{\lambda_{L}\left( {X_{V_{L},A_{N - L}}\left( {m,n} \right)} \right)}^{2} + {\psi_{L}\left( {X_{D_{L},A_{N - L}}\left( {m,n} \right)} \right)}^{2}}\end{matrix}},} \\{{{if}\mspace{14mu} L} < N}\end{matrix} \right.} & \left( {2a} \right)\end{matrix}$where it is possible that some μ_(L), λ_(L), or ψ_(L) are set to zerofor various values of L.

By setting some of these weights, μ_(L), λ_(L) and ψ_(L) to zero some ofthe intermediate and some of the detail subbands of image X areeliminated from the aggregation process. Thus, only selected subbandsthat correspond to non-zero weights are chosen for aggregation. More thenumber of selected subbands potentially more accurate are the results ofthe decomposition. However this additional accuracy comes at the cost ofincreasing the computational overhead. The method of the embodiments ofthis invention strikes an effective tradeoff between accuracy andcomputational complexity and determines the selected intermediatesubbands based on an accuracy to be achieved in determining the measureof quality. The number of the intermediate subbands used determines thecomputational overhead and can be chosen for achieving a given accuracy.A similar process is used in aggregating the detail subbands of image Y.

The steps of the method performing a level by level aggregation of theintermediate and the detail subbands that was described in the previousparagraph are explained with the help of FIG. 3( d). The methodcomprises aggregating one or more of selected detail subbands of theimage X produced at level N producing an edge map at level N for theimage X and aggregating one or more of the selected intermediatesubbands produced at said each level i (i=1 to N−1) producing an edgemap at level i for the image X. The aggregation process terminates afteraggregating the edge maps at all the levels i with edge map at the levelN. The edge map at the level i and the edge map at level N can bedetermined by applying eq. 2(a) whereas the edge map of the image X canbe determined as a weighted sum of the edge maps at the level j and theedge map at the level N for the image X by using eq. 6(a). A similarprocess is followed for aggregating the intermediate subbands and thedetail subbands of the image Y. Upon start (box 352), procedure 350 setsthe value of L to 1 (box 354). A number of steps are then performediteratively for level 1 to N. The value of L displays the current levelof decomposition in this iterative process. A set of predeterminedweights ω_(L), μ_(L), λ_(L), ψ_(L), are obtained (box 356). The edge mapfor the image X at level L and for image Y at level L are then generated(box 358 and box 360). The value of L is incremented by 1 (box 362) andthe procedure 350 checks whether or not L is less than or equal to N(box 364). If so, the procedure 350 exits ‘Yes’ from box 364 and loopsback to the input of box 356 to perform the processing for the nextlevel. Otherwise, the procedure 350 exits ‘No’ from box 364 andaggregates the edge maps for image X for the N levels (box 366) and forthe image Y for the N levels (box 368). After the aggregation isperformed the procedure 350 exits (box 369).

Embodiments of the invention performing cross-layer aggregation arepresented next. In contrast to the level by level method, thecross-layer method performs aggregation across multiple levels. Usingthis approach, generating a horizontal edge map, a vertical edge map anda diagonal edge map of image X and image Y is performed as anintermediate step in the three following embodiments described in thenext paragraph. Generation of such edge maps for image X is discussednext. A similar set of equations describe the generation of such edgemaps for image Y. The intermediate subbands produced at each level i(i=1 to N−1) for the image X include level i-detail subbands, and leveli-wavelet packet (WP) subbands. The level-i detail subbands includelevel i-horizontal, level i-vertical, and level i-diagonal subbands. Thelevel i-wavelet packet subbands in turn include level i-wavelet packetapproximation subbands and level i-wavelet packet detail subbands. Thelevel i-wavelet packet approximation subbands comprise a leveli-horizontal WP approximation subband, a level i-vertical WPapproximation subband and a level i-diagonal WP approximation subband.

The cross layer aggregation involving the detail subbands and thewavelet packet approximation subbands is captured in the following setof equations (10). Note that the summations in these equations areperformed from L=1 to N−1. Each value of L corresponds to a particularlevel i in the N level decomposition process. The terms corresponding toL=1 in eq. 10 corresponds to the first level (i=1) of decomposition, forL=2 to the second level of decomposition (i=2) and so on. Thus, the termlevel L will be used in place of level i in the following discussion.Let:

$\begin{matrix}{{X_{H}\left( {m,n} \right)} = {{\sum\limits_{L = 1}^{N - 1}{\mu_{L}\left( {X_{H_{L},A_{N - L}}\left( {m,n} \right)} \right)}^{2}} + {\mu_{N}\left( {X_{H_{N}}\left( {m,n} \right)} \right)}^{2}}} & \left( {10a} \right) \\{{X_{V}\left( {m,n} \right)} = {{\sum\limits_{L = 1}^{N - 1}{\mu_{L}\left( {X_{V_{L},A_{N - L}}\left( {m,n} \right)} \right)}^{2}} + {\lambda_{N}\left( {X_{V_{N}}\left( {m,n} \right)} \right)}^{2}}} & \left( {10b} \right) \\{{X_{D}\left( {m,n} \right)} = {{\sum\limits_{L = 1}^{N - 1}{\psi_{L}\left( {X_{D_{L},A_{N - L}}\left( {m,n} \right)} \right)}^{2}} + {\psi_{N}\left( {X_{D_{N}}\left( {m,n} \right)} \right)}^{2}}} & \left( {10c} \right)\end{matrix}$

Then we can aggregate them also as one of the three following options:

Option 1:X _(E)(m,n)=√{square root over (X _(H)(m,n))}+√{square root over (X_(V)(m,n))}+√{square root over (X _(D)(m,n))}Option 2:X _(E)(m,n)=√{square root over (X _(H)(m,n)+X _(V)(m,n)+X_(D)(m,n))}{square root over (X _(H)(m,n)+X _(V)(m,n)+X_(D)(m,n))}{square root over (X _(H)(m,n)+X _(V)(m,n)+X _(D)(m,n))}Option 3:X _(E)(m,n)=X _(H)(m,n)+X _(V)(m,n)+X _(D)(m,n)Normally, μ_(L)≦μ_(L+1), λ_(L)≦λ_(L+1), ψ_(L)≦ψ_(L+1)Note also that it is possible to set μ_(k)=0, λ_(l)=0, ψ_(m)=0 for none,one or several values of k, l, m respectively.

X_(H), X_(V), and X_(D) on the left hand sides of equation 10a, 10b and10c are the horizontal, vertical and diagonal edge maps for the image Xrespectively. The terms on the right hand side of each of theseequations include the level L-WP packet approximation subbands and thedetail subbands for the image X. The first term on the right hand sideof equation 10a is a weighted sum of the squares of all the levelL-horizontal WP approximation subbands for the image X, with L rangingfrom 1 to N−1. The second term is a weighted square of the horizontalsubband for the image X obtained after the last level of decomposition.The first term on the right hand side of equation 10b is a weighted sumof the squares of all the level L-vertical WP approximation subbands forthe image X, with L ranging from 1 to N−1. The second term is a weightedsquare of the vertical subband for the image X obtained after the lastlevel of decomposition. The first term on the right hand side ofequation 10c is a weighted sum of the squares of all the levelL-diagonal WP approximation subbands for the image X, with L rangingfrom 1 to N−1. The second term is a weighted sum of the squares of thediagonal subband for the image X obtained after the last level ofdecomposition. The subbands involved in computing the various edge mapsare enclosed in solid double boxes in table 300 and table 302 of FIG. 3(a 1) and FIG. 3( a 2). The predetermined weights ψ_(L), μ_(L), λ_(L) areused to associate varying degrees of importance with each of thesubbands.

After obtaining the horizontal, vertical and diagonal edge maps, thereare three options (Option 1-Option 3) for aggregating (combining) theseedge maps. Three different embodiments of the invention are provided,with each using a particular option.

The steps of the method used to perform the aggregation based on eq. 10and Option 1-Option 3 are explained with the help of flowchart 370presented in FIG. 3( e). Upon start (box 372), the procedure 370 getsthe weights ψ_(L), ψ_(L), λ_(L) associated with each level L=1 to N (box374). The horizontal edge map for image X, X_(H) is generated next (box376). This is followed by the generation of the vertical edge map forthe image X, X_(V) (box 378) and of the diagonal edge map for the imageX, X_(D) (box 380). Eq 10(a), eq. 10(b) and eq. 10(c) are used for thegeneration of each of these edge maps. The aggregation of the three edgemaps for the image X is then performed (box 382) and the edge map forthe image X, X_(E), is produced. This is followed by the generation ofthe horizontal edge map for the image Y, Y_(H), (box 384), the verticaledge map for the image Y, Y_(V) (box 386) and the diagonal edge map forthe image Y, Y_(D) (box 388). The procedure 370 then aggregates thethree edge maps for the image Y (box 390) producing the edge map for theimage Y, Y_(E), and exits (box 392). Note that each of the threeembodiments perform the aggregation in box 390 in a different way byusing one of the three options, Option 1-Option 3.

As shown in the eq.10 only the wavelet packet approximation subbands arechosen from the intermediate subbands to perform the aggregation.Although using only the wavelet packet approximation subbands contributeto reducing computational complexity, one does not need to limit theaggregation process to include only the wavelet packet approximationsubbands X_(H) _(L) _(,A) _(N-L) , X_(V) _(L) _(,A) _(N-L) , and X_(D)_(L) _(,A) _(N-L) . The wavelet packet detail subbands, comprising leveli-WP horizontal, a level i-WP vertical and level i-WP diagonal subbandscan be included in the aggregation process as well. Additionalembodiments of the invention including such wavelet packet detailsubbands are used if one needs to include more details in thehorizontal, vertical and diagonal directions.

For these embodiments the aggregation process takes the following formcaptured in eq. 11. Let:

$\begin{matrix}{{X_{H}^{\prime}\left( {m,n} \right)} = {{\sum\limits_{L = 1}^{N - 1}{\mu_{L}\left( {X_{H_{L},A_{N - L}}\left( {m,n} \right)} \right)}^{2}} + {\mu_{N}\left( {X_{H_{N}}\left( {m,n} \right)} \right)}^{2} + {\sum\limits_{L = 1}^{N - 1}{\mu_{L}^{\prime}\left( {X_{H_{L},H_{N - L}}\left( {m,n} \right)} \right)}^{2}}}} & \left( {11a} \right) \\{{X_{V}^{\prime}\left( {m,n} \right)} = {{\sum\limits_{L = 1}^{N - 1}{\lambda_{L}\left( {X_{V_{L},A_{N - L}}\left( {m,n} \right)} \right)}^{2}} + {\lambda_{N}\left( {X_{V_{N}}\left( {m,n} \right)} \right)}^{2} + {\sum\limits_{L = 1}^{N - 1}{\lambda_{L}^{\prime}\left( {X_{V_{L},V_{N - L}}\left( {m,n} \right)} \right)}^{2}}}} & \left( {11b} \right) \\{{X_{D}^{\prime}\left( {m,n} \right)} = {{\sum\limits_{L = 1}^{N - 1}{\psi_{L}\left( {X_{D_{L},A_{N - L}}\left( {m,n} \right)} \right)}^{2}} + {\psi_{N}\left( {X_{D_{N}}\left( {m,n} \right)} \right)}^{2} + {\sum\limits_{L = 1}^{N - 1}{\psi_{L}^{\prime}\left( {X_{D_{L},D_{N - L}}\left( {m,n} \right)} \right)}^{2}}}} & \left( {11c} \right)\end{matrix}$

Then we can aggregate them also as one of the three following options:

Option 1a:X _(E)(m,n)=√{square root over (X′ _(H)(m,n))}+√{square root over (X′_(V)(m,n))}+√{square root over (X′ _(D)(m,n))}Option 2a:X _(E)(m,n)=√{square root over (X′ _(H)(m,n)+X′ _(V)(m,n)+X′_(D)(m,n))}{square root over (X′ _(H)(m,n)+X′ _(V)(m,n)+X′_(D)(m,n))}{square root over (X′ _(H)(m,n)+X′ _(V)(m,n)+X′ _(D)(m,n))}Option 3a:X _(E)(m,n)=X′ _(H)(m,n)+X′ _(V)(m,n)+X′ _(D)(m,n)Normally, μ_(L)≦μ_(L+1), λ_(L)≦λ_(L+1), ψ_(L)≦ψ_(L+1), μ′_(L)≦μ′_(L+1),λ′_(L)≦λ′_(L+1), ψ′_(L≦ψ′) _(L+1)Note also that it is possible to set μ_(k)=0, λ₁=0, ψ_(m)=0, μ′_(k′)=0,λ′_(l′)=0, ψ′_(m′)=0 for none, one or several values of k, l, m, k′, l′,m′ respectively.

A method similar to the one described in FIG. 3( e) is used. Thedifference is that eq. 11a, eq, 11b and eq. 11c are used in thecomputation of the horizontal, vertical and diagonal edge maps of theimage X and the image Y. Contributions of the level L-WP horizontalsubbands for the image X, with L ranging from 1 to N−1 are captured bythe last term on the right hand side of eq. 11a. Similarly, the lastterm on the right hand side of eq. 11b captures the contributions of thelevel L-WP vertical subbands and the last term on the right hand side ofeq. 11c captures the contributions of the level L-WP diagonal subbandsof the image X. The level L-WP horizontal, vertical and diagonalsubbands, with L ranging from 1 to N−1, used in computing the variousedge maps are enclosed in boxes with dotted lines in table 300 and table302 of FIG. 3( a 1) and FIG. 3( a 2). Once again, a different embodimentis available for performing the aggregation of the horizontal, verticaland diagonal edge maps by using one of the three options, Option1a-Option 3a.

As mentioned earlier, various IQMs are used in conjunction with theframework described earlier lead to multiple embodiments, one for eachIQM. These IQMs include SSIM, VIF, PSNR and AD. Each of theseembodiments is discussed next.

Embodiment Based on the SSIM IQM

This embodiment uses the SSIM IQM. The steps of the SSIM-based methodfor this embodiment of the invention are described with the help of FIG.4. Upon start (box 402), the procedure 400 sets N, the number of levelsfor the DWT to 1 (box 404). Since the image approximation subband playsthe major role in the embodiments of the invention, N is to be chosen insuch a way that it maximizes the accuracy of the approximation qualityscore SSIM_(A) the computation of which is described in the nextparagraph. The plots in graph 500 presented in FIG. 5( a) show thelinear correlation coefficient (LCC) and Spearman's rank correlationcoefficient (SRCC) between SSIM_(A) and the differential mean opinionscore (DMOS) values for different values of N. In performing this test,all 779 distorted images of the LIVE Image Quality Assessment DatabaseRelease 2 described by H. R. Sheikh, Z. Wang, L. Cormack, and A. G.Bovik in “LIVE Image Quality Assessment Database Release 2”, availableat: http://live.ece.utexas.edu/research/quality were included incomputing the LCC and the SRCC. As can be seen from FIG. 5( a), SSIM_(A)attains its best performance for N=1. The reason is that for more thanone level of decomposition, the approximation subband becomes verysmall, and a number of important image structures are lost in theapproximation subband.

A single level (N=1) DWT using the Haar filter is performed on both thereference (image X) and distorted (image Y) images (box 406). The methodcan be adapted to handle other types of wavelet filters as well. In thenext step, the procedure 400 computes SSIM_(A), the SSIM between theapproximation subbands of image X and image Y (box 408). For each imagepatch x_(A) and y_(A) within the first level approximation subbands of Xand Y, SSIM_(A) is computed by using eq. (12):SSIM_(A)(x _(A) ,y _(A))=SSIM(x _(A) ,y _(A))  (12)

The SSIM map is calculated according to the method described by Z. Wang,A. Bovik, H. Sheikh, and E. Simoncelli, in Image quality assessment:From error visibility to structural similarity, IEEE Transactions onImage Processing, vol. 13, no. 4, pp. 600-612, April 2004.

All the parameters are kept the same as those proposed in this paper,except the window size. A sliding 4×4 Gaussian window is used byprocedure 400.

Procedure 400 then generates the edge maps for image X and image Y (box410). The edge map is produced for each image by using eq.13 and eq. 14.

$\begin{matrix}{{X_{E}\left( {m,n} \right)} = \sqrt{{0.45 \cdot \left( {X_{H_{1}}\left( {m,n} \right)} \right)^{2}} + {0.45 \cdot \left( {X_{V_{1}}\left( {m,n} \right)} \right)^{2}} + {0.1 \cdot \left( {X_{D_{1}}\left( {m,n} \right)} \right)^{2}}}} & (13) \\{{Y_{E}\left( {m,n} \right)} = \sqrt{{0.45 \cdot \left( {Y_{H_{1}}\left( {m,n} \right)} \right)^{2}} + {0.45 \cdot \left( {Y_{V_{1}}\left( {m,n} \right)} \right)^{2}} + {0.1 \cdot \left( {Y_{D_{1}}\left( {m,n} \right)} \right)^{2}}}} & (14)\end{matrix}$where (m,n) shows the sample position within the wavelet subbands.Please note that eq. 12 and eq. 13 are based on eq. 2 and eq. (3)described earlier. The weights used are: μ=0.45, λ=0.45 and ψ=0.1. Inthe next step, SSIM_(E), the edge SSIM map between two images iscalculated (box 412) using the following equation:

$\begin{matrix}{{{SSIM}_{E}\left( {x_{E},y_{E}} \right)} = \frac{{2\sigma_{x_{E},y_{E}}} + c}{\sigma_{x_{E}}^{2} + \sigma_{y_{E}}^{2} + c}} & (15) \\{{c = ({kL})^{2}},{k = 1}} & (16)\end{matrix}$where σ_(x) _(E) _(,y) _(E) is the covariance between image patchesx_(E) and y_(E) (of X_(E) and Y_(E)); parameters σ_(X) _(E) ² and σ_(y)_(E) ² are variances of x_(E) and y_(E) respectively; k is a smallconstant; and L is a dynamic range of pixels (255 for gray-scaleimages). The correlation coefficient and variances are computed in thesame manner as presented by Z. Wang, A. Bovik, H. Sheikh, and E.Simoncelli, in “Image quality assessment: From error visibility tostructural similarity”, IEEE Transactions on Image Processing, vol. 13,no. 4, pp. 600-612, April 2004. In fact, as the edge map only formsimage-edge structures and contains no luminance information, theluminance comparison part of the SSIM map described in this paper isomitted for determining the edge SSIM map. The contrast map is generatednext (box 414) by using eq. (4). The contrast map is used for weightedpooling of SSIM_(A) and SSIM_(E) (box 416) by using equations (7) and(8). Note that IQM_(A) used with eq. (7) and IQM_(E) used with eq. (8)are SSIM_(A) (computed by eq. 12) and SSIM_(E) (computed by eq. (15))respectively. A SSIM_(DWT) score is determined as the measure of quality(box 418) by using eq. (9):SSIM_(DMT)(X,Y)=β·S _(A)+(1−β)·S _(E)  (17)

When the root mean square (RMSE) between the SSIM_(DWT) and DMOS valuesare computed for different β values in eq. (17), it reaches its globalminimum for 0=0.87. This value of β meets the intuitive expectation thatβ should be close to 1. FIG. 5( b) shows graph 520, which presents theplot of RMSE for different values of β. Please note that LCC has lowsensitivity to small variations in β. That is, the proposed β=0.87 doesnot significantly affect the accuracy of the SSIM_(DWT) for measuringquality of images in a different image database.

After generating the measure of quality the procedure 400 exits (box420).

Embodiment Based on the VIF IQM

The VIF index is observed to be one of the most accurate image qualitymetrics in the performance evaluation of prominent image qualityassessment algorithms described by H. R. Sheikh, M. F. Sabir, and A. C.Bovik, in “A statistical evaluation of recent full reference imagequality assessment algorithms”, IEEE Transactions on Image Processing,vol. 15, no. 11, pp. 3440-3451, November 2006. In spite of its highlevel of accuracy, this IQM has not been given as much consideration asthe SSIM IQM in a variety of applications. This is probably because ofits high computational complexity (6.5 times the computation time of theSSIM index according to H. R. Sheikh, and A. C. Bovik, in “Imageinformation and visual quality”, IEEE Transactions on Image Processing,vol. 15, no. 2, pp. 430-444, February 2006). Most of the complexity inthe VIF IQM comes from over-complete steerable pyramid decomposition, inwhich the neighboring coefficients from the same subband are linearlycorrelated. Consequently, the vector Gaussian scale mixture GSM isapplied for accurate quality prediction.

This section, explains the steps of the method for calculating VIF inthe discrete wavelet domain by exploiting the proposed framework. Theproposed approach is more accurate than the original VIF index-basedapproach, and yet is less complex in comparison to the VIF IQM. Itapplies real Cartesian-separable wavelets and uses a scalar GSM insteadof vector GSM in modeling the images for VIF computation. A shortexplanation is provided next.

Scalar GSM-Based VIF:

Scalar GSM has been described and applied in the computation of IFC anddiscussed by H. R. Sheikh, A. C. Bovik, and G. de Veciana, in “Aninformation fidelity criterion for image quality assessment usingnatural scene statistics”, IEEE Transactions on Image Processing, vol.14, no. 12, pp. 2117-2128, December 2005. That procedure is repeatedhere for VIF index calculation using scalar GSM. In FIG. 6( a), box 602represents the GSM model of the undistorted original image (signal) andbox 604 considers image distortions as a channel distortion and addsdistortion to the original signal from box 602 to model the distortedimage signal. Boxes 606 and 608 show the function of HVS as additivewhite Gaussian noise and their outputs are the perceived distorted andoriginal signals, respectively. Considering FIG. 6( a), let C^(M)=(C₁,C₂, . . . , C_(M)) denote M elements from C, and let D^(M)=(D₁, D₂, . .. , D_(m)) be the corresponding M elements from D. C. and D denote therandom fields (RFs) from the reference and distorted signalsrespectively. As in described by H. R. Sheikh, A. C. Bovik, and G. deVeciana, in “An information fidelity criterion for image qualityassessment using natural scene statistics”, IEEE Transactions on ImageProcessing, vol. 14, no. 12, pp. 2117-2128, December 2005, the modelscorrespond to one subband. C is a product of two stationary randomfields (RFs) that are independent of each other:C={C _(i) :iεI}=S·U={S _(i) ·U _(i) :iεI}  (18)where i denotes the set of spatial indices for the RF, S is an RF ofpositive scalars, and U is a Gaussian scalar RF with mean zero andvariance σ_(U) ². The distortion model is a signal attenuation andadditive Gaussian noise, defined as follows:D={D _(i) :iεI}=GC+V={g _(i) C _(i) +V _(i) :iεI}  (19)where G is a deterministic scalar attenuation field, and V is astationary additive zero-mean Gaussian noise RF with variance σ_(V) ².The perceived signals in FIG. 6( a) described by H. R. Sheikh, and A. C.Bovik, in “Image information and visual quality”, IEEE Transactions onImage Processing, vol. 15, no. 2, pp. 430-444, February 2006, aredefined as:E=C+N, F=D+N′  (20)where N and N′ represent stationary white Gaussian noise RFs withvariance σ_(N) ². If the steps outlined by in this paper for VIF indexare used in the calculation considering scalar GSM one obtains:

$\begin{matrix}{{I\left( {C^{M};{{E^{M}❘S^{M}} = s^{M}}} \right)} = {{I\left( {C^{M};{E^{M}❘s^{M}}} \right)} = {\frac{1}{2}{\sum\limits_{i = 1}^{M}{\log_{2}\left( \frac{{s_{i}^{2}\sigma_{U}^{2}} + \sigma_{N}^{2}}{\sigma_{N}^{2}} \right)}}}}} & (21)\end{matrix}$

In the GSM model, the reference image coefficients are assumed to havezero mean. So, for the scalar GSM model, estimates of can be obtained bylocalized sample variance estimation. As discussed by H. R. Sheikh, A.C. Bovik, and G. de Veciana, in “An information fidelity criterion forimage quality assessment using natural scene statistics”, IEEETransactions on Image Processing, vol. 14, no. 12, pp. 2117-2128,December 2005, the variance σ_(U) ² can be assumed to be unity withoutloss of generality. Thus, eq. (21) is simplified to eq. (22).

$\begin{matrix}{{I\left( {C^{M};{E^{M}❘s^{M}}} \right)} = {\frac{1}{2}{\sum\limits_{i = 1}^{M}{\log_{2}\left( {1 + \frac{\sigma_{C_{i}}^{2}}{\sigma_{N}^{2}}} \right)}}}} & (22)\end{matrix}$

Similarly, eq. (23) is obtained:

$\begin{matrix}{{I\left( {C^{M};{F^{M}❘s^{M}}} \right)} = {\frac{1}{2}{\sum\limits_{i = 1}^{M}{\log_{2}\left( {1 + \frac{g_{i}^{2}\sigma_{C_{i}}^{2}}{\sigma_{V}^{2} + \sigma_{N}^{2}}} \right)}}}} & (23)\end{matrix}$

The final VIF index is obtained from eq. (23), by using the methoddescribed by H. R. Sheikh, and A. C. Bovik, in “Image information andvisual quality”, IEEE Transactions on Image Processing, vol. 15, no. 2,pp. 430-444, February 2006, but considering only a single subband:

$\begin{matrix}{{VIF} = \frac{I\left( {C^{M};{F^{M}❘s^{M}}} \right)}{I\left( {C^{M};{E^{M}❘s^{M}}} \right)}} & (24)\end{matrix}$

The steps of the VIF-based method for this embodiment of the inventionis described with the help of flowchart 650 FIG. 6( b). Upon start (box652), the procedure 650 sets the value of N=1 (box 654). As explainedbefore in the case of the embodiment based on the SSIM IQM, the rightnumber of decomposition levels for calculating the VIF-based measure ofquality needs to be determined first. Experiments are performed in a waysimilar to the experiments described earlier for scalar VIF. FIG. 7( a)shows graph 700 presenting the LCC and SRCC between VIF_(A) (thecomputation of which is described in the next paragraph) and the DMOSvalues for various decomposition levels N. It can be seen that theVIF_(A) prediction accuracy decreases as the number of decompositionlevels increases. Therefore, VIF_(A) performance is best at N=1. This isbecause the resolution of approximation subband is reduced by multileveldecomposition, and consequently the image information, on which VIF isbased, will be lost in that subband for higher values of N.

A N level DWT is performed on image X and image Y next (box 656).Procedure 650 then computes the VIF approximation quality score,VIF_(A), between the approximation subbands of X and Y, i.e. X_(A) andY_(A). For simplicity, X_(A) and Y_(A) are used here instead of X_(A) ₁and Y_(A) ₁ :

$\begin{matrix}{{VIF}_{A} = \frac{\sum\limits_{i = 1}^{M}{\log_{2}\left( {1 + \frac{g_{i}^{2}\sigma_{x_{A,i}}^{2}}{\sigma_{V_{i}}^{2} + \sigma_{N}^{2}}} \right)}}{\sum\limits_{i = 1}^{M}{\log_{2}\left( {1 + \frac{\sigma_{x_{A,i}}^{2}}{\sigma_{N}^{2}}} \right)}}} & (25)\end{matrix}$where M is the number of samples in the approximation subband, x_(A,i)is the ith image patch within the approximation subband X_(A), and σ_(x)_(A,i) ² is the variance of. The noise variance σ_(N) ² is set to 5 inthis embodiment of the invention. The parameters g_(i) and σ_(V) _(i) ²are estimated as described by H. R. Sheikh, A. C. Bovik, and G. deVeciana, in “An information fidelity criterion for image qualityassessment using natural scene statistics”, IEEE Transactions on ImageProcessing, vol. 14, no. 12, pp. 2117-2128, December 2005, resulting ineq. (26) and eq. (27).

$\begin{matrix}{g_{i} = \frac{\sigma_{x_{A,i},y_{A,i}}}{\sigma_{x_{A,i}}^{2} + ɛ}} & (26)\end{matrix}$where σ_(x) _(A,i) _(,y) _(A,i) is the covariance between image patchesx_(A,i) and y_(A,i), and ε is a very small constant to avoid instabilitywhen σ_(x) _(A,i) ² is zero and ε=10⁻²⁰.σ_(V) _(i) ²=σ_(y) _(A,i) ² −g _(i)·σ_(x) _(A,i) _(,y) _(A,i)   (27)

All the statistics (the variance and covariance of image patches) arecomputed within a local Gaussian square window, which movespixel-by-pixel over the entire approximation subbands X_(A) and Y_(A).The Gaussian sliding window used in this case is exactly the same asthat defined earlier. Because of the smaller resolution of the subbandsin the wavelet domain, one can even extract reasonably accurate localstatistics with a small 3×3 sliding window. But to reach the bestperformance and extracting accurate local statistics, a larger window ofsize 9×9 is exploited here. Simulation results described in a latersection show that the measure of quality based on VIF can provideaccurate scores with the set up described here.

In the next step, the procedure 650 generates the edge maps X_(E) andY_(E) using eq. (2) and eq. (3) (box 660). The VIF edge quality score,VIF_(E), between the edge maps of the image X and the image Y thatreflects the similarity between the edge maps is computed next (box662). Finally, the measure of quality is computed by using eq. (9) (box664):VIF_(DWT)(X,Y)=β·VIF_(A)+(1=β)·VIF_(E)0<β≦1  (28)where VIF_(DWT) score is the measure of quality for image Y in the range[0,1]. After computing the measure of quality, the procedure 650 exits(box 666).

FIG. 7( b) presents graph 720 that shows the RMSE between the VIF_(DwT)and DMOS values for different values of β. It is observed that RMSEreaches its global minimum at β=0.81. This value of β is close to theone obtained for SSIM_(DWT). It is notable that LCC does not have highsensitivity to small variations in β. Thus, the obtained value for β isstill valid if VIF_(DWT) is applied on images in a different imagedatabase.

Embodiment Based on the PSNR IQM

This embodiment uses the PSNR IQM for determining the measure of qualityfor the distorted image. The conventional PSNR and its equivalent, meansquare error (MSE), are defined in eq. (29) and eq. (30).

$\begin{matrix}{{{PSNR}\left( {X,Y} \right)} = {10 \cdot {\log_{10}\left( \frac{X_{\max}^{2}}{{MSE}\left( {X,Y} \right)} \right)}}} & (29) \\{{{MSE}\left( {X,Y} \right)} = {\frac{1}{N_{P}} \cdot {\sum\limits_{m,n}\left( {{X\left( {m,n} \right)} - {Y\left( {m,n} \right)}} \right)^{2}}}} & (30)\end{matrix}$where X and Y denote the reference and distorted images respectively.X_(max) is the maximum possible pixel value of the reference image X.The minimum pixel value is assumed to be zero. N_(P) is the number ofpixels in each of the images. Although the conventional PSNR is stillpopular because of its ability to easily compute quality in decibels(dB), it cannot adequately reflect the human perception of imagefidelity. Other error-based techniques, such as WSNR and NQM discussedby N. Damera-Venkata, T. D. Kite, W. S. Geisler, B. L. Evans, and A. C.Bovik, in “Image quality assessment based on a degradation model” inIEEE Transactions on Image Processing, vol. 9, no. 4, pp. 636-650, April2000 and VSNR discussed by D. M. Chandler, and S. S. Hemami, in “Awavelet-based visual signal-to-noise ratio for natural images”, IEEETransactions on Image Processing, vol. 16, no. 9, pp. 2284-2298,September 2007, are less simple to use, as they follow sophisticatedprocedures to compute the human visual system (HVS) parameters. Thissection explains how to calculate a PSNR-based measure of qualityaccurately in the discrete wavelet domain by using the frameworkprovided by the embodiment of the invention described earlier.

The steps of the PSNR-based method for this embodiment of the inventionare described with the help of FIG. 8. Upon start (box 802), theprocedure 800 computes N, the number of levels for the DWT (box 804).The plots in graph 900 displayed in FIG. 9( a) show the LCC and SRCCbetween PSNR_(A) and the DMOS values for different decomposition levels.The test is done in a way similar to that described in the previoussections. It can be seen that PSNR_(A) attains its best and near bestperformance for N=2 and N=3. Based on individual types of distortion,which are available in the corresponding image database, one candetermine which value of N (2 or 3) provides more accurate measures ofquality. Table I denoted by reference numeral 960 and shown in FIG. 9(c) lists the SRCC values for five different types of distortion. It isobserved that the performance of PSNR_(A) (the computation of which isdescribed later in this section) is superior with N=2 when all data(distorted images) is considered. Thus, N=2 is the appropriatedecomposition level.

Although N=2 works fine for the entire image database used in thetesting, a N that can be adapted to various other images needs be usedin this embodiment. The number of levels of decomposition N isdetermined as a function of a minimum size of the approximation subband,S, which produces a substantially peak response for human visual system.Thus eq. 1(e) is used for determining N.

After the value of N is computed a N level DWT is applied to image X(box 806) and to image Y (box 808). A N level discrete wavelet transform(DWT) on both the reference and distorted images is applied by usingbased on the Haar filter. The method can handle other wavelet filters aswell. With N level decomposition, the approximation subbands X_(A) _(N), and Y_(A) _(N) , as well as a number of detail subbands, are obtained.As discussed earlier, the Haar filter is chosen for its modestcomputational simplicity and good performance. This simplicity imposesnegligible computational burden on the over all method of theembodiments of the invention. As indicated by the simulation results,the Haar wavelet also provides more accurate quality scores than otherwavelet bases.

In the next step, the procedure 800 generates the edge maps for theimage X (box 810) and for the image Y (box 812) next. Then, theprocedure 800 computes the PSNR approximation quality score PSNR_(A) byapplying the PSNR IQM between the approximation subbands of the image Xand the image Y (box 814). The PSNR edge quality score PSNR_(E) iscomputed next by applying the PSNR IQM between the edge maps of theimage X and the image Y (box 816). PSNR_(A) and PSNR_(E) are computed byusing eq. (30) and eq. (31).PSNR_(A)=PSNR(X _(A) _(N) ,Y _(A) _(N) )  (30)PSNR_(E)=PSNR(X _(E) ,Y _(E))  (31)

In the next step, the procedure 800 determines a PSNR_(DWT) score as themeasure of quality (box 818) and exits (box 820). PSNR_(DWT) isdetermined by combining the PSNR approximation quality score and thePSNR edge quality score according to eq. (9):

$\begin{matrix}{{{{PSNR}_{DWT}\left( {X,Y} \right)} = {{\beta \cdot {PSNR}_{A}} + {\left( {1 - \beta} \right) \cdot {PSNR}_{E}}}}{0 < \beta \leq 1}} & (32)\end{matrix}$

Please note that PSNR_(DWT) gives the measure of quality for variousimages in an image database. To find the optimum value of the constantβ, an experiment similar to those described in the previous subsectionsfocusing on SSIM_(DWT) and VIF_(DWT) is performed. FIG. 9( b) presentsgraph 920 that shows the RMSE between the PSNR_(DWT) and DMOS values fordifferent values of β used in eq. (32). It is observed that RMSE isglobally minimum at β=0.84. This value of β is quite close to thosevalues determined earlier for other metrics. Verification of this valueof β performing well on the other test databases is discussed in a latersection of this document.

Embodiment Based on the AD IQM

This embodiment uses the AD IQM. The steps of the AD-based method forthis embodiment of the invention are described with the help of FIG. 10.Upon start (box 1002), the procedure 1000 determines N (box 1004). Atest was performed on the image database mentioned earlier. FIG. 11presents graph 1100 showing the LCC and SRCC between an approximation ADmap, AD_(A), (the generation of which is described in the nextparagraph) and the DMOS values for different decomposition levels thatare obtained. Similar to the PSNR_(A), the AD_(A) performance is best atN=2. The general value of N can be calculated by eq. (1e).

A N level DWT is performed on image X and image Y next (box 1006). Inthe following step, the procedure 1000 applies the AD IQM between theapproximation subbands of X and Y to produce the approximation AD map,AD_(A), (box 1008).AD_(A)(m,n)=|X _(A) _(N) (m,n)−Y _(A) _(N) (m,n)|  (33)where (m,n) shows a sample position in the approximation subband.

The edge maps are then generated for the image X and the image Y (box1010). Next, the procedure 1000 applies the AD IQM between the edge mapsX_(E) and Y_(E) to produce the edge AD map, AD_(E), (box 1012).AD_(E)(m,n)=|X_(E)(m,n)−Y _(E)(m,n)|  (34)

After generating the edge AD map, the procedure 1000 generates thecontrast map (box 1014). The contrast map is obtained by using eq. (4),and then AD_(A) and AD_(E) are pooled using the contrast map todetermine approximation quality score and the edge quality score S_(A)and S_(E) respectively (box 1016).

$\begin{matrix}{S_{A} = \frac{\sum\limits_{j = 1}^{M}\mspace{14mu}{{Contrast}\mspace{14mu}{\left( {m,n} \right) \cdot {{AD}_{A}\left( {m,n} \right)}}}}{\sum\limits_{j = 1}^{M}\mspace{14mu}{{Contrast}\mspace{14mu}\left( {m,n} \right)}}} & (35) \\{S_{E} = \frac{\sum\limits_{j = 1}^{M}\mspace{14mu}{{Contrast}\mspace{14mu}{\left( {m,n} \right) \cdot {{AD}_{E}\left( {m,n} \right)}}}}{\sum\limits_{j = 1}^{M}\mspace{14mu}{{Contrast}\mspace{14mu}\left( {m,n} \right)}}} & (36)\end{matrix}$

The method uses the S_(A) and the S_(E) to determine an AD_(DWT) scoreas the measure of quality (box 1018) by using eq. (37).

$\begin{matrix}{{{{AD}_{DWT}\left( {X,Y} \right)} = {{\beta \cdot S_{A}} + {\left( {1 - \beta} \right) \cdot S_{E}}}}{0 < \beta \leq 1}} & (37)\end{matrix}$

As discussed in the previous section β is set to 0.84. After determiningthe measure of quality, the procedure 1000 exits (box 1020).

Simulation Results

The performance of the methods of embodiments of this invention thatdetermines the measure of quality for a distorted image are evaluated byusing the images in the LIVE Image Quality Assessment Database, Release2 described by H. R. Sheikh, Z. Wang, L. Cormack, and A. C. Bovik, in“LIVE Image Quality Assessment Database Release 2,” available athttp://live.ece.utexas.edu/research/quality. This database includes 779distorted images derived from 29 original colour images using five typesof distortion: JPEG compression, JPEG2000 compression, Gaussian whitenoise (GWN), Gaussian blurring (GBlur), and the Rayleigh fast fading(FF) channel model. The realigned subjective quality data for thedatabase are used in all experiments.

The three different IQM-based measures of quality, each computed by anembodiment described earlier are applied to the images in the databaseto measure the performance of the objective models. A number of metricsare used in this performance evaluation. The first metric is the Pearsoncorrelation coefficient (LCC) between the Difference Mean Opinion Score(DMOS) and the objective model outputs after nonlinear regression. Thecorrelation coefficient gives an evaluation of prediction accuracy. Thefive-parameter logistical function defined by H. R. Sheikh, M. F. Sabir,and A. C. Bovik, in “A statistical evaluation of recent full referenceimage quality assessment algorithms”, IEEE Transactions on ImageProcessing, vol. 15, no. 11, pp. 3440-3451, November 2006 for nonlinearregression is used. The second metric is the root mean square error(RMSE) between the DMOS and the objective model outputs after nonlinearregression. The RMSE is considered as a measure of predictionconsistency. The third metric is the Spearman rank correlationcoefficient (SRCC), which provides a measure of prediction monotonicity.

In order to put the performance evaluation of the embodiments of thisinvention into the proper context, the methods of the embodiments arecompared to other image quality metrics available in the prior art,including the conventional PSNR, the spatial domain mean SSIM discussedby Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image qualityassessment: From error visibility to structural similarity”, IEEE Trans.Image Process., vol. 13, no. 4, pp. 600-612, April 2004, an autoscaleversion of SSIM performing downsampling on images discussed by Z. Wangin Z. Wang's SSIM Research Homepage athttp://www.ece.uwaterloo.ca/˜z70wang/research/ssim/, and a WSNRdiscussed by N. Damera-Venkata, T. D. Kite, W. S. Geisler, B. L. Evans,and A. C. Bovik, in “Image quality assessment based on a degradationmodel” in IEEE Transactions on Image Processing, vol. 9, no. 4, pp.636-650, April 2000., in which the images are filtered by the CSFspecified by J. L. Mannos, and D. J. Sakrison, in “The effects of avisual fidelity criterion on the encoding of images”, IEEE Trans. Inf.Theory, vol. IT-20, no. 4, pp. 525-536, July 1974 and by H. R. Sheikh,Z. Wang, L. Cormack, and A. C. Bovik, in “LIVE Image Quality AssessmentDatabase Release 2,” available at:http://live.ece.utexas.edu/research/quality and in Z. Wang's SSIMResearch Homepage cited above.

Table II 1200 in FIG. 12 lists LCC between different metrics and DMOSvalues of the LIVE database and Table III 1220 in FIG. 13 and Table IV1240 in FIG. 14 show RMSE and SRCC between the metrics and DMOS valuesrespectively. It is observed that the SRCC of mean SSIMA from 0.9441increases to 0.9573 for SA, which signifies a 1.32% of improvement. Theperformance of SSIM_(DWT) is the best among all structural metrics. ForSNR-based metrics, the SRCC of PSNR_(A) is 0.9307 which is higher thanconventional PSNR (0.8756) and even WSNR (0.9240), while its complexityis lower than the conventional PSNR.

The embodiments of the invention provide a number of advantagesincluding the following. An important contribution of the invention isthat a contrast map in the wavelet domain is introduced for poolingquality maps. The new approach of the current invention does not requireheavy experimentation to determine parameters, and it is adaptive todifferent wavelet filters based on a contrast map defined for pooling aSSIM quality map for example. Various changes and modifications of theembodiments shown in the drawings and described in the specification maybe made within the scope of the following claims without departing fromthe scope of the invention in its broader aspect. For example, insteadof a Haar transform, other wavelet transforms such as Newland transform,or a wavelet transform using a Daubechies filter can be used in themulti-resolution decomposition of the reference and distorted images.Various steps of methods including method 250 of FIG. 2( b) may beperformed in parallel by using a multi-core CPU or a multiprocessorsystem. For example, the decomposition of the images X and Y during themultiresolution decomposition can be performed in parallel. Suchparallel computations can reduce the computation time for calculatingthe measure of quality.

Although the embodiments of the invention have been described in detail,it will be apparent to one skilled in the art that variations andmodifications to the embodiment may be made within the scope of thefollowing claims.

What is claimed is:
 1. A method for determining a measure of quality fora distorted image Y, characterizing a similarity between the image Y andan undistorted reference image X, having the same number of rows andcolumns of pixels as the image Y, the method comprising: (a1) applyingan N level multiresolution decomposition, comprising levels 1, 2, . . .i, i+1, . . . N, to the image X, to produce: for each level i, with iranging from 1 to N−1, intermediate subbands of image X for processingat level i+1; and for the level N, an approximation subband containingmain content of the image X and detail subbands containing edges of theimage X; (b1) applying said N level multiresolution decomposition,comprising levels 1, 2, . . . i, i+1, . . . N, to the image Y, toproduce: for each level i, with i ranging from 1 to N−1, intermediatesubbands of image Y for processing at level i+1; and for the level, N,an approximation subband containing main content of the image Y anddetail subbands containing edges of the image Y; (c1) applying an imagequality metric (IQM) to the approximation subband of the image X and theapproximation subband of the image Y to produce an approximation qualitymeasure characterizing similarity between the main content of the imageX and the main content of the image Y; (d1) aggregating the intermediatesubbands at the level i for the image X, with i ranging from levels 1 toN−1, and the detail subbands of the image X to produce an edge map ofthe image X characterizing the edges of the image X; (e1) aggregatingthe intermediate subbands at the level i for the image Y, with i rangingfrom 1 to N−1, and the detail subbands of the image Y to produce an edgemap of the image Y characterizing the edges of image Y; (f1) applyingthe IQM between the edge map of the image X and the edge map of theimage Y to produce an edge quality measure characterizing similaritybetween the edges of the image X and the edges of the image Y; and (g1)processing the approximation quality measure and the edge qualitymeasure to determine the measure of quality.
 2. The method of claim 1,wherein: (a2) the step (d1) further comprises: (a2i) selecting theintermediate subbands at each level i, with i ranging from 1 to N−1, andthe detail subbands for the image X based on an accuracy to be achievedin determining the measure of quality; and (a2ii) aggregating onlyselected intermediate subbands and selected detail subbands for theimage X; and (b2) the step (e1) further comprises: (b2i) selecting theintermediate subbands at said each level i and the detail subbands forthe image Y based on said accuracy to be achieved in determining themeasure of quality; and (b2ii) aggregating only selected intermediatesubbands and selected detail subbands for the image Y.
 3. The method ofclaim 2, wherein the step (a2i) and the step (b2i) further compriseselecting the intermediate subbands at said each level i and the detailsubbands for the image X and the image Y based on a number of theintermediate and the detailed subbands required for achieving theaccuracy.
 4. The method of claim 3, further comprising: (a4) at the step(a2ii): (a4i) for said each level i, aggregating the selectedintermediate subbands for the image X producing an edge map at the leveli for the image X; (a4ii) aggregating the selected detail subbands forthe image X producing an edge map at the level N for the image X; and(a4iii) aggregating the edge map at said each level i for the image Xwith the edge map at the level N for the image X; and (b4) at the step(b2ii): (b4i) for said each level i, aggregating the selectedintermediate subbands for the image Y producing an edge map at the leveli for the image Y; (b4ii) aggregating the selected detail subbands forthe image Y producing an edge map at the level N for the image Y; and(b4iii) aggregating the edge map at said each level i for the image Ywith the edge map at the level N for the image Y.
 5. The method of claim4, wherein: (a5) the step (a4iii) further comprises: (a5i) multiplyingthe edge map at said each level i for the image X with a predeterminedweight for the level i; (a5ii) multiplying the edge map at the level Nfor the image X with a predetermined weight at the level N; and (a5iii)summing products resulting from the multiplying in the step (a5i) and aproduct resulting from the multiplying in the step (a5ii); and (b5) thestep (b4iii) further comprises: (b5i) multiplying the edge map at saideach level i for the image Y with the predetermined weight for the leveli; (b5ii) multiplying the edge map at the level N for the image Y withthe predetermined weight at the level N; and (b5iii) summing productsresulting from the multiplying in the step (b5i) and a product resultingfrom the multiplying in the step (b5ii).
 6. The method of claim 5,wherein: (a6) the step (a4i) comprises: (a6i) for said each level i,squaring each selected intermediate subband for the image X; (a6ii)multiplying a result of the squaring performed in the step (a6i) with apredetermined weight for said each selected intermediate subband for theimage X; (a6iii) summing products resulting from the multiplyingperformed in the step (a6ii); (a6iv) applying a square root function toa result of the summing performed in the step (a6iii) producing the edgemap at the level i for the image X; (b6) the step (a4ii) comprises:(b6i) squaring each said selected detail subband for the image X; (b6ii)multiplying a result of the squaring performed in the step (b6i) with apredetermined weight for said each selected detail subband for the imageX; (b6iii) summing products resulting from the multiplying performed inthe step (a6ii); (b6iv) applying a square root function to a result ofthe summing performed in the step (a6iii) producing the edge map at thelevel N for the image X; (c6) the step (b4i) comprises: (c6i) for saideach level i, squaring each selected intermediate subband for the imageY; (c6ii) multiplying a result of the squaring performed in the step(c6i) with a predetermined weight for said each selected intermediatesubband for the image Y; (c6iii) summing products resulting from themultiplying performed in the step (c6ii); (c6iv) applying a square rootfunction to a result of the summing performed in the step (c6iii)producing the edge map at the level i for the image Y; and (d6) the step(b4ii) comprises: (d6i) squaring each said selected detail subband forthe image Y; (d6ii) multiplying a result of the squaring performed inthe step (d6i) with a predetermined weight for said each selected detailsubband for the image Y; (d6iii) summing products resulting from themultiplying performed in the step (d6ii); (d6iv) applying a square rootfunction to a result of the summing performed in the step (d6iii)producing the edge map at the level N for the image Y.
 7. The method ofclaim 3, wherein the intermediate subbands at said each level i for theimage X and the image Y include for each respective image: leveli-detail subbands and level i-wavelet packet (WP) subbands; the leveli-WP subbands comprising level i-WP approximation subbands and leveli-WP detail subbands; and the level i-WP approximation subbands furthercomprising a level i-horizontal WP approximation subband, a leveli-vertical WP approximation subband and a level i-diagonal WPapproximation subband.
 8. The method of claim 7, wherein: (a8) the step(a2i) further comprises selecting one or more of the level i-WPapproximation subbands and one or more of the detail subbands for theimage X; and (b8) the step (b2i) further comprises selecting one or moreof the level i-WP approximation subbands and one or more of the detailsubbands for the image Y.
 9. The method of claim 8, wherein: (a9) thestep (a8) further comprises including one or more of the level i-detailsubbands for the image X in the selecting; and (b9) the step (b8)further comprises including one or more of the level i-detail subbandsfor the image Y in the selecting.
 10. The method of claim 9, wherein:(a10) the step (a2ii) further comprises: (a10i) aggregating the leveli-horizontal WP approximation subband for the image X at said each leveli and the horizontal subband for the image X producing a horizontal edgemap for the image X; (a10ii) aggregating the level i-vertical WPapproximation subband for the image X at said each level i and thevertical subband for the image X producing a vertical edge map for theimage X; (a10iii) aggregating the level i-diagonal WP approximationsubband for the image X at said each level i and the diagonal subbandfor the image X producing a diagonal edge map for the image X; and(a10iv) aggregating the horizontal edge map, the vertical edge map andthe diagonal edge map for the image X; and (b10) the step (b2ii) furthercomprises: (b10i) aggregating the level i-horizontal WP approximationsubband for the image Y at said each level i and the horizontal subbandfor the image Y producing a horizontal edge map for the image Y; (b10ii)aggregating the level i-vertical WP approximation subband for the imageY at said each level i and the vertical subband for the image Yproducing a vertical edge map for the image Y; (b10iii) aggregating thelevel i-diagonal WP approximation subband for the image Y at said eachlevel i and the diagonal subband for the image Y producing a diagonaledge map for the image Y; and (b10iv) aggregating the horizontal edgemap, the vertical edge map and the diagonal edge map for the image Y.11. The method of claim 10, wherein the level i-WP detail subbands forthe image X and the image Y further comprise for each respective image,a level i-WP horizontal subband, a level i-WP vertical subband and alevel i-WP diagonal subband and: (a11) the step (a10i) further comprisesincluding the level i-WP horizontal subband for the image X in theaggregating; (b11) the step (a10ii) further comprises including thelevel i-WP vertical subband for the image X in the aggregating; (c11)the step (a10iii) further comprises including the level i-WP diagonalsubband for the image X in the aggregating; (d11) the step (b10i)further comprises including the level i-WP horizontal subband for theimage Y in the aggregating; (e11) the step (b10ii) further comprisesincluding the level i-WP vertical subband for the image Y in theaggregating; and (f11) the step (b10iii) further comprises including thelevel i-WP diagonal subband for the image Y in the aggregating.
 12. Themethod of claim 10, wherein: (a12) the step (a10iv) comprises: (a12)applying a square root function to the horizontal edge map, the verticaledge map and the diagonal edge map for the image X; and (b12) summingresults of applying said square root function; and (b12) the step(b10iv) comprises: (a12) applying a square root function to thehorizontal edge map, the vertical edge map and the diagonal edge map forthe image Y; and (b12) summing results of applying said square rootfunction.
 13. The method of claim 11, wherein: (a13) the step (a10iv)comprises: (a13i) summing the horizontal edge map, the vertical edge mapand the diagonal edge map for the image X; and (a13ii) applying a squareroot function to result of the summing performed in the step (a13i); and(b13) the step (b10iv) comprises: (b13i) summing the horizontal edgemap, the vertical edge map and the diagonal edge map for the image Y;and (b13ii) applying a square root function to result of the summingperformed in the step (b13i).
 14. The method of claim 1, wherein thesteps (a1) and (b1) comprise applying an N level discrete wavelettransform (DWT).
 15. The method of claim 14, wherein the discretewavelet transform is a Haar transform.
 16. The method of claim 14,wherein the discrete wavelet transform is one of a Newland transform, orwavelet transform using a Daubechies filter.
 17. The method of claim 3,comprising one or more of the following: (a17) at the step (a1): (a17i)for the level 1, applying the multiresolution decomposition to the imageX producing intermediate subbands at the level 1 for processing at level2; and (a17ii) for the level i, with i ranging from 2 to N, applying themultiresolution decomposition to one or more of the intermediatesubbands produced by the multiresolution decomposition performed at thelevel i−1; and (b17) at the step (b1): (b17i) for the level 1, applyingthe multiresolution decomposition to the Image Y producing intermediatesubbands at the level 1 for processing at level 2; and (b17ii) for thelevel i, with i ranging from 2 to N, applying the multiresolutiondecomposition to one or more of the intermediate subbands produced bythe multiresolution decomposition performed at the level i−1.
 18. Themethod of claim 2, wherein the approximation quality measure is anapproximation quality map, and the edge quality measure is an edgequality map, and the step (g1) further comprises: (a18) generating acontrast map, including assigning corresponding values to the pixels ofthe approximation subband and the edge map of the image X and the imageY according to their respective importance to human visual system; (b18)performing weighted pooling of the approximation quality map using thecontrast map to produce an approximation quality score; (c18) performingweighted pooling of the edge quality map using the contrast map toproduce an edge quality score; and (d18) combining the approximationsimilarity score from the step (b18) with the edge similarity score fromthe step (c18) to determine the measure of quality.
 19. The method ofclaim 1, further comprising the step of determining N as a function of aminimum size of an approximation subband, S, which produces asubstantially peak response for human visual system.
 20. The method ofclaim 1, wherein: (a20) the step (c1) comprises applying a structuralsimilarity (SSIM) IQM to the approximation subband of the image X andthe approximation subband of the image Y to produce an approximationSSIM map, SSIM_(A); (b20) the step (f1) comprises applying the SSIM IQMbetween the edge map of the image X and the edge map of the image Y toproduce an edge SSIM map, SSIM_(E); and (c20) the step (g1) comprisesprocessing the SSIM_(A) and the SSIM_(E) to determine a SSIM_(DWT) scoreas the measure of quality.
 21. The method of claim 1, wherein: (a21) thestep (c1) comprises applying an Absolute Difference (AD) IQM to theapproximation subband of the image X and the approximation subband ofthe image Y to produce an approximation AD map, AD_(A); (b21) the step(f1) comprises applying the AD IQM between the edge map of the image Xand the edge map of the image Y to produce an edge AD map, AD_(E); and(c21) the step (g1) comprises processing the AD_(A) and the AD_(E) todetermine an AD_(DWT) score as the measure of quality.
 22. The method ofclaim 1, wherein: (a22) the step (c1) comprises applying apeak-signal-to-noise ratio (PSNR) IQM to the approximation subband ofthe image X and the approximation subband of the image Y to produce aPSNR approximation quality score, PSNR_(A); (b22) the step (f1)comprises applying the PSNR IQM between the edge map of the image X andthe edge map of the image Y to produce a PSNR edge quality score,PSNR_(E); and (c22) the step (g1) comprises processing the PSNR_(A) andthe PSNR_(E) to determine a PSNR_(DWT) score as the measure of quality.23. The method of claim 1, wherein: (a23) the step (c1) comprisesapplying a Visual Information Fidelity (VIF) IQM to the approximationsubband of the image X and the approximation subband of the image Y toproduce a VIF approximation quality score, VIF_(A); (b23) the step (f1)comprises applying the VIF IQM between the edge map of the image X andthe edge map of the image Y to produce a VIF edge quality score,VIF_(E); and (c23) the step (g1) comprises processing the VIF_(A) andthe VIF_(E) to determine a VIF_(DWT) score as the measure of quality.24. The method of claim 2, wherein: (a24) the step (a2) furthercomprises choosing the selected intermediate subbands at said each leveli and the detail subbands for the image X, having substantially sameresolution, for aggregating; and (b24) the step (b2) further compriseschoosing the selected intermediate subbands at said each level i and thedetail subbands for the image Y, having substantially same resolution,for aggregating.
 25. A system for determining a measure of quality for adistorted image Y, characterizing a similarity between the image Y andan undistorted reference image X, having the same number of rows andcolumns of pixels as the image Y, the system comprising: a processor,and a computer readable storage medium having computer readableinstructions stored thereon, which, when executed by the processor, formthe following: (a25) a first decomposition unit applying an N levelmultiresolution decomposition, comprising levels 1, 2, . . . i, i+1, . .. N, to the image X, to produce: for each level i, with i ranging from 1to N−1, intermediate subbands of image X for processing at level i+1;and for the level N, an approximation subband containing main content ofthe image X and detail subbands containing edges of the image X; (b25) asecond decomposition unit applying said N level multiresolutiondecomposition, comprising levels 1, 2, . . . i, i+1, . . . N, to theimage Y, to produce: for each level i, with i ranging from 1 to N−1,intermediate subbands for image Y; and for the level, N, anapproximation subband containing main content of the image Y and detailsubbands containing edges of the image Y; (c25) an approximation qualitymeasure unit applying an image quality metric (IQM) to the approximationsubband of the image X and the approximation subband of the image Y toproduce an approximation quality measure characterizing similaritybetween the main content of the image X and the main content of theimage Y; (d25) a first aggregation unit aggregating the intermediatesubbands at the level i for the image X, with i ranging from levels 1 toN−1 and the detail subbands of the image X to produce an edge map of theimage X characterizing the edges of the image X; (e25) a secondaggregation unit aggregating the intermediate subbands at the level ifor the image Y, with i ranging from 1 to N−1 and the detail subbands ofthe image Y to produce an edge map of the image Y characterizing theedges of image Y; (f25) an edge quality measure unit applying the IQMbetween the edge map of the image X and the edge map of the image Y toproduce an edge quality measure characterizing similarity between theedges of the image X and the edges of the image Y; and (g25) a qualitymeasure unit processing the approximation quality measure and the edgequality measure to determine the measure of quality.
 26. The system ofclaim 25, wherein: (a26) the first aggregation unit (d25) furthercomprises: (a26i) a first selection unit selecting the intermediatesubbands at each level i, with i ranging from 1 to N−1, and the detailsubbands for the image X based on an accuracy to be achieved indetermining the measure of quality; and (a26ii) a first selectiveaggregation unit aggregating only selected intermediate subbands andselected detail subbands for the image X; and (b26) the secondaggregation unit (e1) further comprises: (b26i) a second selection unitselecting the intermediate subbands at said each level i and the detailsubbands for the image Y based on said accuracy to be achieved indetermining the measure of quality; and (b26ii) a second selectiveaggregation unit aggregating only selected intermediate subbands andselected detail subbands for the image Y.
 27. The system of claim 26,wherein: (a27) the first selection unit (a26i) further comprises a firstselection sub-unit selecting the intermediate subbands at said eachlevel i and the detail subbands for the image X based on a number of theintermediate and the detailed subbands required for achieving theaccuracy; and (b27) the second selection unit (b26i) further comprises asecond selection sub-unit selecting the intermediate subbands at saideach level i and the detail subbands for the image Y based on a numberof the intermediate and the detailed subbands required for achieving theaccuracy.
 28. The system of claim 27, further comprising: (a28) at thefirst selective aggregation unit (a26ii): (a28i) a first leveli-intermediate subbands aggregation unit aggregating, for said eachlevel i, the selected intermediate subbands for the image X producing anedge map at the level i for the image X; (a28ii) a first detail subbandsaggregation unit aggregating the selected detail subbands for the imageX producing an edge map at the level N for the image X; and (a28iii) afirst edge map determination unit aggregating the edge map at said eachlevel i for the image X with the edge map at the level N for the imageX; and (b28) at the second selective aggregation unit (b26ii): (b28i) asecond level i-intermediate subbands aggregation unit aggregating, forsaid each level i, the selected intermediate subbands for the image Yproducing an edge map at the level i for the image Y; (b28ii) a seconddetail subbands aggregation unit aggregating the selected detailsubbands for the image Y producing an edge map at the level N for theimage Y; and (b28iii) a second edge map determination unit aggregatingthe edge map at said each level i for the image Y with the edge map atthe level N for the image Y.
 29. The system of claim 28, wherein: (a29)the first edge map determination unit (a28iii) further comprises a firstcomputation unit for: multiplying the edge map at said each level i forthe image X with a predetermined weight for the level i; multiplying theedge map at the level N for the image X with a predetermined weight forthe level N; and summing products resulting from the multiplying theedge map at said each level for the image X and the multiplying the edgemap at the level N for the image X; (b29) the second edge mapdetermination unit (b28iii) further comprises a second computation unitfor: multiplying the edge map at said each level i for the image Y withthe predetermined weight for the level i; multiplying the edge map atthe level N for the image Y with the predetermined weight for the levelN; and summing products resulting from the multiplying the edge map atsaid each level for the image Y and the multiplying the edge map at thelevel N for the image Y.
 30. The system of claim 29, wherein: (a30) thefirst level i-intermediate subbands aggregation unit (a28i) comprises afirst level i-computation unit for: squaring each selected intermediatesubband for the image X for said each level i; multiplying a result ofthe squaring with a predetermined weight for said each selectedintermediate subband for the image X; summing products resulting fromthe multiplying; and applying a square root function to a result of thesumming producing the edge map at the level i for the image X; (b30) thefirst detail subbands aggregation unit (a28ii) comprises a first detailsubbands computation unit for: squaring each said selected detailsubband for the image X; multiplying a result of the squaring with apredetermined weight for said each selected detail subband for the imageX; summing products resulting from the multiplying; and applying asquare root function to a result of the summing producing the edge mapat the level N for the image X; (c30) the second level i-intermediatesubbands aggregation unit (b28i) comprises a second level i-computationunit for: squaring each said selected intermediate subband for the imageY for said each level i; multiplying a result of the squaring performedwith a predetermined weight for said each selected intermediate subbandfor the image Y; summing products resulting from the multiplying;applying a square root function to a result of the summing producing theedge map at the level i for the image Y; and (d30) the second detailsubbands aggregation unit (b28ii) comprises a second detail subbandscomputation unit for: squaring each said selected detail subband for theimage Y; multiplying a result of the squaring with a predeterminedweight for said each selected detail subband for the image Y; summingproducts resulting from the multiplying; and applying a square rootfunction to a result of the summing producing the edge map at the levelN for the image Y.
 31. The system of claim 27, wherein the intermediatesubbands at said each level i for the image X and the image Y includefor each respective image: level i-detail subbands and level i-waveletpacket (WP) subbands; the level i-WP subbands comprising level i-WPapproximation subbands and level i-WP detail subbands; and the level-WPapproximation subbands further comprising a level i-horizontal WPapproximation subband, a level i-vertical WP approximation subband and alevel i-diagonal WP approximation subband.
 32. The system of claim 31,wherein: (a32) the first selection unit (a26i) further comprises a firstWP-detail selection sub-unit selecting one or more of the level i-WPapproximation subbands and one or more of the detail subbands for theimage X; and (b32) the second selection unit step (b26i) furthercomprises a second WP-detail selection sub-unit selecting one or more ofthe level i-WP approximation subbands and one or more of the detailsubbands for the image Y.
 33. The system of claim 32, wherein: (a33) thefirst WP-detail selection sub-unit (a32) further comprises a firstdetail-selection module including one or more of the level i-detailsubbands for the image X in the selecting; and (b33) the secondWP-detail selection sub-unit (b32) further comprises a seconddetail-selection module including one or more of the level i-detailsubbands for the image Y in the selecting.
 34. The system of claim 33,wherein: (a34) the first selective aggregation unit (a26ii) furthercomprises: (a34i) a first horizontal edge map unit aggregating the leveli-horizontal WP approximation subband for the image X at said each leveli and the horizontal subband for the image X producing a horizontal edgemap for the image X; (a34ii) a first vertical edge map unit aggregatingthe level i-vertical WP approximation subband for the image X at saideach level i and the vertical subband for the image X producing avertical edge map for the image X; (a34iii) a first diagonal edge mapunit aggregating the level i-diagonal WP approximation subband for theimage X at said each level i and the diagonal subband for the image Xproducing a diagonal edge map for the image X; and (a34iv) a firstcombination unit aggregating the horizontal edge map, the vertical edgemap and the diagonal edge map for the image X; and (b34) the secondselective aggregation unit (b26ii) further comprises: (b34i) a secondhorizontal edge map unit aggregating the level i-horizontal WPapproximation subband for the image Y at said each level i and thehorizontal subband for the image Y producing a horizontal edge map forthe image Y; (b34ii) a second vertical edge map unit aggregating thelevel i-vertical WP approximation subband for the image Y at said eachlevel i and the vertical subband for the image Y producing a verticaledge map for the image Y; (b34iii) a second diagonal edge map unitaggregating the level i-diagonal WP approximation subband for the imageY at said each level i and the diagonal subband for the image Yproducing a diagonal edge map for the image Y; and (b34iv) a secondcombination unit aggregating the horizontal edge map, the vertical edgemap and the diagonal edge map for the image Y.
 35. The system of claim34, wherein the level i-WP detail subbands for the image X and the imageY further comprise for each respective image, a level i-WP horizontalsubband, a level i-WP vertical subband and a level i-WP diagonalsubband; and (a35) the first horizontal edge map unit (a34i) furthercomprises a first horizontal sub-unit including the level i-WPhorizontal subband for the image X in the aggregating; (b35) the firstvertical edge map unit (a34ii) further comprises a first verticalsub-unit including the level i-WP vertical subband for the image X inthe aggregating; (c35) the first diagonal edge map unit (a34iii) furthercomprises a first vertical sub-unit including the level i-WP diagonalsubband for the image X in the aggregating; (d35) the second horizontaledge map unit (b34i) further comprises a second horizontal sub-unitincluding the level i-WP horizontal subband for the image Y in theaggregating; (e35) the second vertical edge map unit (b34ii) furthercomprises a second vertical sub-unit including the level i-WP verticalsubband for the image Y in the aggregating; and (f35) the seconddiagonal edge map unit (b34iii) further comprises a second diagonalsub-unit including the level i-WP diagonal subband for the image Y inthe aggregating.
 36. The system of claim 34, wherein: (a36) the firstcombination unit (a34iv) comprises a first combination sub-unit for:applying a square root function to the horizontal edge map, the verticaledge map and the diagonal edge map for the image X; and summing resultsof applying said square root function; and (b36) the second combinationunit (b34iv) comprises a second combination sub-unit for: applying asquare root function to the horizontal edge map, the vertical edge mapand the diagonal edge map for the image Y; and summing results ofapplying said square root function.
 37. The system of claim 35, wherein:(a37) the first combination unit (a34iv) comprises a third combinationsub-unit for: summing the horizontal edge map, the vertical edge map andthe diagonal edge map for the image X; and applying a square rootfunction to result of the summing; and (b37) the second combination unit(b34iv) comprises a fourth combination sub-unit for: summing thehorizontal edge map, the vertical edge map and the diagonal edge map forthe image Y; and applying a square root function to result of thesumming.
 38. The system of claim 25, wherein the first decompositionunit (a1) and the second decomposition unit (b1) comprise computationalmeans for applying an N level discrete wavelet transform (DWT).
 39. Thesystem of claim 38, wherein the discrete wavelet transform is a Haartransform.
 40. The system of claim 38, wherein the discrete wavelettransform is one of a Newland transform, or wavelet transform using aDaubechies filter.
 41. The system of claim 27, comprising one or more ofthe following: (a41) at the first decomposition unit (a25): (a41i) afirst level 1 decomposition unit applying the multiresolutiondecomposition to the image X producing intermediate subbands at thelevel 1 for processing at level 2; (a41ii) a first level i decompositionunit applying the multiresolution decomposition to one or more of theintermediate subbands produced by the multiresolution decompositionperformed at the level i−1, with i ranging from 2 to N; (b41) at thesecond decomposition unit (b25): (b41i) a second level 1 decompositionunit applying the multiresolution decomposition to the Image Y producingintermediate subbands at the level 1 for processing at level 2; (b41ii)a second level i decomposition unit applying the multiresolutiondecomposition to one or more of the intermediate subbands produced bythe multiresolution decomposition performed at the level i−1, with iranging from 2 to N.
 42. The system of claim 26, wherein theapproximation quality measure is an approximation quality map, and theedge quality measure is an edge quality map, and the quality measureunit (g25) further comprises: (a42) a contrast map unit generating acontrast map, including assigning corresponding values to the pixels ofthe approximation subband and the edge map of the image X and the imageY according to their respective importance to human visual system; (b42)a first pooling unit performing weighted pooling of the approximationquality map using the contrast map to produce an approximation qualityscore; (c42) a second pooling unit performing weighted pooling of theedge quality map using the contrast map to produce an edge qualityscore; and (d42) a score combination unit combining the approximationsimilarity score from the step (b42) with the edge similarity score fromthe step (c42) to determine the measure of quality.
 43. The system ofclaim 25, further comprising computational means for determining N as afunction of a minimum size of the approximation subband, S, whichproduces a substantially peak response for human visual system.
 44. Thesystem of claim 25, wherein: (a44) the approximation quality measureunit (c25) comprises an approximation SSIM map unit applying a SSIM IQMto the approximation subband of the image X and the approximationsubband of the image Y to produce an approximation structural similarity(SSIM) map, SSIM_(A); (b44) the edge quality measure unit (f25)comprises an edge SSIM map unit applying the SSIM IQM between the edgemap of the image X and the edge map of the image Y to produce an edgeSSIM map, SSIM_(E); and (c44) the quality measure unit (g25) comprises aSSIM_(DWT) score unit processing the SSIM_(A) and the SSIM_(E) todetermine a SSIM_(DWT) score as the measure of quality.
 45. The systemof claim 25, wherein: (a45) the approximation quality measure unit (c25)comprises an approximation AD map unit applying an AD IQM to theapproximation subband of the image X and the approximation subband ofthe image Y to produce an approximation AD map, AD_(A); (b45) the edgequality measure unit (f25) comprises an edge AD map unit applying the ADIQM between the edge map of the image X and the edge map of the image Yto produce an edge AD map, AD_(E); and (c45) the quality measure unit(g25) comprises an AD_(DWT) score unit processing the AD_(A) and theAD_(E) to determine an AD_(DWT) score as the measure of quality.
 46. Thesystem of claim 25, wherein: (a46) the approximation quality measureunit (c25) comprises a PSNR approximation quality score unit applying aPSNR IQM to the approximation subband of the image X and theapproximation subband of the image Y to produce a PSNR approximationquality score, PSNR_(A); (b46) the edge quality measure unit (f25)comprises a PSNR edge quality score determination unit applying the PSNRIQM between the edge map of the image X and the edge map of the image Yto produce a PSNR edge quality score, PSNR_(E); and (c46) the qualitymeasure unit (g25) comprises a PSNR_(DWT) score unit processing thePSNR_(A) and the PSNR_(E) to determine a PSNR_(DWT) score as the measureof quality.
 47. The system of claim 25, wherein: (a47) the approximationquality measure unit (c25) comprises a VIF approximation quality scoreunit applying a VIF IQM to the approximation subband of the image X andthe approximation subband of the image Y to produce a VIF approximationquality score, VIF_(A); (b47) the edge quality measure unit (f25)comprises a VIF edge quality score determination unit applying the VIFIQM between the edge map of the image X and the edge map of the image Yto produce a VIF edge quality score, VIF_(E); and (c47) the qualitymeasure unit (g25) comprises a VIF_(DWT) score unit processing theVIF_(A) and the VIF_(E) to determine a VIF_(DWT) score as the measure ofquality.
 48. The system of claim 26, wherein: (a48) the firstaggregation unit (a26) further comprises computational means forchoosing the selected intermediate subbands at said each level i and thedetail subbands for the image X, having substantially same resolution,for aggregating; and (b48) the second aggregation unit (b26) furthercomprises computational means for choosing the selected intermediatesubbands at said each level i and the detail subbands for the image Y,having substantially same resolution, for aggregating.
 49. Anon-transitory computer readable storage medium, having computerreadable program code instructions stored thereon, which, when executedby a processor, perform the following: (a49) applying an N levelmultiresolution decomposition, comprising levels 1, 2, . . . i, i+1, . .. N, to the image X, to produce: for each level i, with i ranging from 1to N−1, intermediate subbands of image X for processing at level i+1;and for the level N, an approximation subband containing main content ofthe image X and detail subbands containing edges of the image X; (b49)applying said N level multiresolution decomposition, comprising levels1, 2, . . . i, i+1, . . . N, to the image Y, to produce: for each leveli, with i ranging from 1 to N−1, intermediate subbands of image Y forprocessing at level i+1; and for the level, N, an approximation subbandcontaining main content of the image Y and detail subbands containingedges of the image Y; (c49) applying an image quality metric (IQM) tothe approximation subband of the image X and the approximation subbandof the image Y to produce an approximation quality measurecharacterizing similarity between the main content of the image X andthe main content of the image Y; (d49) aggregating the intermediatesubbands at the level i for the image X, with i ranging from levels 1 toN−1, and the detail subbands of the image X to produce an edge map ofthe image X characterizing the edges of the image X; (e49) aggregatingthe intermediate subbands at the level i for the image Y, with i rangingfrom 1 to N−1, and the detail subbands of the image Y to produce an edgemap of the image Y characterizing the edges of image Y; (f49) applyingthe IQM between the edge map of the image X and the edge map of theimage Y to produce an edge quality measure characterizing similaritybetween the edges of the image X and the edges of the image Y; and (g49)processing the approximation quality measure and the edge qualitymeasure to determine the measure of quality.